SRE-Filter provides a number of useful features including:
This document describes how to use SRE-Filter.
While not necessary,
the reader should first examine
the documentation that comes with GoServe (GOSERVE.DOC). Knowledge of the
REXX
programming language would also help. However, you can still do a lot of nice stuff with only a
rudimentary understanding of GoServe, or of REXX. Lastly, if you don't know
HTML, the
"markup language" of
the world-wide web (WWW), then it's going to be hard to do anything interesting!
Before reading this manual, we strongly recommend that you peruse the SRE-Filter users guide -- it's a much easier read!
Other useful documentation files include:
A popular alternative to this filter is Don Meyer's GoHTTP HTTP filter system.
GOHTTP offers CGI-BIN, HTACCESS compatability, common-log auditing, and rudimentary support for server side includes.Another alternative is the InterFilt filter of Derek Sims. It has good support for multiple realms and multiple hosts, but lacks server side includes and other features.
filter
program files
(SREFILTR.80, SREFMON.CMD, and others), as well as
a set of files containing external routines and
samples of initialization databases.
As noted above, you'll need the
GoServe
package (ver 2.50 is recommended).
UNZIP SREFV12I)
.
From an OS/2 prompt, run INSTALL (a REXX program
bundled with SRE-Filter),
and follow the on-line instructions. You will be asked to provide the
names of several directories.
If you select the defaults, INSTALL will create the following directories (you select which drive):
After running INSTALL, you can delete the contents of this temporary directory.
filter
to be SREFILTR.80. To do this,
you can insert http filter srefiltr.80 in the
optional parameters line in GoServe's settings window
(right-mouse click on the GoServe icon ...)
Limits
:
End client after inactive
End client after total
DataDir
option points to the directory containing your
HTML files (for example, \WWW).
call
filter anyways
option.
The most straightforward way of doing this is:
SREFILTR.80 and INITFILT.80
optional parameters
setting of GoServe.
:nnn
in the IP address of your server.
If you change the names of your directories, keep the following in mind:
Of greater import, SRE-Filter uses several "helper" threads to handle username lookup, alias matching, SSI-document caching, SEL-specific access control, virtual directory matches, and post-filter processing. These threads are generated upon the first request to SRE-Filter. In addition, SRE-Filter stores a number of variables in the local (GoServe) environment.
For most users, these implementation details are unimportant. However, these threads do use system resources (queues) which remain alive even after GoServe has been turned off. To free these resources, it is recommended that the FREEUP.CMD program be run after GoServe has been killed.
Citation: SRE-Filter uses the REXXLIB library to access macrospace and to generate/communicate between threads. A distribution copy of REXXLIB is supplied with SRE-Filter, and may be used freely with SRE-Filter. If you wish to use REXXLIB in other REXX programs, contact Quercus Systems for licensing information.Lastly, SRE-Filter monitors the parameter files, including INITFILT.80. Changes to these files will be incorporated into SRE-Filter's parameters about every 10 seconds.
In the interest of thoroughness, this manual discusses the multitude of SRE-Filter options in terms of directly modifying these parameters. However, most individuals will probably never need to actually edit a file. Instead, almost all of the parameters mentioned herein can be modified using the intermediate-mode configurator (CONFIG2.HTM). In fact, in most cases the simple-mode configurator (CONFIG0.HTM) will do the job -- where appropriate, links to the appropriate section of the simple-mode configurator will be made available for your browsing pleasure .If you are a hands-on kind of administrator, feel free to directly modify these parameters. If so, you might also want to read the detailed descripton of INITFILT.80.
In many cases, the connection between the selector and server resource it refers to is simple (such as when the selector is a filename relative to the GoServe data directory). In other cases, it's completely virtual (as when the resource consists of the output of a program that's run using information provided by the client).
The selector is a pointer to a web-server resource. Typically, this would be a file; but it might refer to the output of some program run just for this request. In either case, SRE-Filter has to match the selector to the actual resource (an activity that can become rather convoluted!).In much the same way that OS/2 extended attributes contain additional information about a file, SRE-Filter's SEL-specific information contain additional information about a selector (needless to say, this information is only used by SRE-Filter).
You can use the simple-mode configurator to create these host-specific entries.
The HOSTS. stem variable (set in INITFILT.80) defines the hosts (either IP addresses or aliases) that your server will work with. Each HOST. variable should contain the following information (in a comma delimited list):
HOSTS.n ='ip_address, host_nickname,default_dir'Where:
ip_address
is a IP address (integer or numeric)
host_nickname
is the "nickname" you will use when
referring to this host. The host_nickname
must
not
be a number, it must contain at least one non-digit character. Thus, a
host nickname of 1234
is invalid,
but 1234X
is okay.
default_dir
is the default directory for this host (it
is used instead of the GoServe default data directory).
Thus, if all requests (regardless of intended host) have identical access to your server's resources (it's files and programs) you can ignore this entire section!
Given that you do want to have different resources available to different hosts, almost all of the SRE-Filter variables and parameters can be specified on a "host specific" basis
ZOOSITE// TIGERS/ D:\ANIMALS\TIGERS
where ZOOSITE is the host nickname for ZOO.OURISP.NET, and a space separates the
ZOOSITE// from TIGERS/.
CHECKLOG.ZOOSITE='NO'
refers to this "ZOOSITE" host.
You can use the simple-mode configurator to create these PUBLIC_URLSs.The PUBLIC_URLS. stem variable is used to identify server resources that are "open to the public" -- that can be accessed by clients who have no logon rights and no access privileges.
Basically, SRE-Filter compares the request (using wildcard matching with substitution) against the entries in the PUBLIC_URLS. stem variable.
All requests that match a PUBLIC_URLS stem variable:
PUBLICFILES PUBLIC
,
Notes:
The simple-mode configurator can be used to enable LOGON, to add users to the USER_FILE, to add/remove IN-HOUSE clients, and to enable the LOGON_FAIL_FILE.
Be aware that ...
CHECKLOG=ALWAYS
, the client will only need to enter his
username and password once (that is, she won't need to reenter it each
time a different document is requested).
Note that the OWNERS and INHOUSEIPS.n
stem variables are checked first. If the
client's numeric IP address appears in either of these lists,
UNALLOWEDIPS.n is not checked. Thus, by setting
UNALLOWEDIPS.1="*.*.*.*"
, you can deny entry to everyone
except OWNERS and IN-HOUSE users.
Also note that UNALLOWEDIPS. entries must
be "host-specific" (see INITFILT.DOC for details).
ALLOW_ACCESS='NO'
) requires that the
client have appropriate privileges.
In a sense, privileges offer a convenient shorthand for identifying sets of users.Privileges are set automatically for:
In addition, IN-HOUSE clients and OWNERS will be granted the privileges that are listed in the INHOUSE_PRIVS "space delimited list" variable. It is recommended that INHOUSE_PRIVS always include the INHOUSE privilege.For everyone else:
The simple-mode configurator can be used to enable selector-specific access controls, and to add/remove selector-specific access control entriesThere may be cases where you want clients to have unencumbered access to some, but not to all, the resources of your server. One way to do this is to identify which request selectors a client has access to. The ACCESS_FILE, and the ALLOW_ACCESS variable, are used to construct these selector-specific (SEL-specific) access controls.
ALLOW_ACCESS can take the following values:
Permissions
The permissions list includes
information on:
The Realm
When ALLOW_ACCESS is enabled, and the client does not have
appropriate privileges, SRE-Filter will query the client for a new
username/password. When querying, the SEL-specific realm
is displayed if available. If there is no SEL-specific realm, the
THE_REALM realm is displayed.
ACCESS_FAIL_FILE
If the client does not have access to the server resources represented
by the selector, SRE-Filter will use
the value of ACCESS_FAIL_FILE to dictate the response. If
ACCESS_FAIL_FILE=0, a username/password will be re-requested. Otherwise,
the ACCESS_FAIL_FILE, or a SEL-specific access failure file, will be
returned. For details, see the discusion of ACCESS_FAIL_FILE in
INITFILT.DOC.
Advanced Options
The SEL-specific advanced options file
allows you to specify several SEL-specific advanced options, including:
For security reasons, PUT and DELETE requests will only be honored if
the appropriate permssion (PUT or DELETE
respectively) appears in the permissions list .
Furthermore, when using the PUT_FILE or
GET_URL facilities to upload to a non-upload_directory subdirectory (that is,
to a virtual directory), the PUT permission must apply (to the "relative file
name" that matched this virtual directory).
Note that for caching to occur, the GoServe caching option must be turned on!
* NO
-- if not explicitily mentioned, do NOT allow access
* *
-- if not explicitily mentioned, allow access
Briefly, the HTACCESS method utilizes special HTACCESS
files that are placed
in your various directories (usually with the name HTACCESS.
,
or .HTACCESS
). These files (there may one in each of several
directories) contain entries pointing to password files, user-group files,
and other parameters that
are used to ascertain who shall be given access to the
files in the directory (and it's child directories).
HTACCESS control will be checked for all requests.
Note that the presumed location of
the HTACCESS file depends on the type of request:
on speed of throughput grounds we recommend using the "native" SRE-Filter methods (that is, the SEL-specific method) whenever possible.
/
, or
"/?some+options"
,
SRE-Filter assumes that the client desires the "home page" (the default document).
By tradition (but not necessarily) the default document is a file named
INDEX.HTM located
in the root of the data directory.
The default can be set using the DEFAULT or DEFAULT.HOST_NICKNAME
variable
(that is, DEFAULT may be host specific).
You can use the simple-mode configurator to set the home page and the directory-specific default document(s)
COOLJUNK/JOKES/
does not specify a particular
file, but instead implies "give me the default document for the
COOLJUNK/JOKES subdirectory of the data directory". In these cases,
the AUTO_NAME variable is examined.
AUTO_NAME should contain a space delimited list of filenames to use when a request for a "directory" is recieved. SRE-Filter will attempt to find these files in this "requested" directory -- starting with the first name in the AUTO_NAME list. To assist in this search, a special name, consisting of *.HTM (or *.HTML) means use the directory_name.htm file.
Lastly, an entry of !CREATE instructs SRE-Filter to generate a directory listing of files and subdirectories in the requested directory. This listing is generated with SRE-Filter's !DIR special action. By default, a <PRE> formatted table is generated, containing (for each file):
AUTO_NAME=" *.HTM INDEX.HTM "
/INFO/MAINE/
,
and your data directory is D:\WWW
, then ...
*.HTM
is interpreted as: D:\WWW\INFO\MAINE\MAINE.HTM
INDEX.HTM
is interpreted as: D:\WWW\INFO\MAINE\INDEX.HTM
auto_name=" *.HTML INDEX.HTML PUBLIC_HTML/INDEX.HTML "
(note that PUBLIC_HTML/INDEX.HTM will look in the PUBLIC_HTML directory relative
to the current URL).
DIR
: Interpret as a directory; / is added and
AUTO_NAME is used.
For example, CHARITIES/HEALTH
is interpreted as a sloppy form of
CHARITIES/HEALTH/
(the "CHARITIES/HEALTH subdirectory of the data directory, or a
"local or remote" virtual directory).
REDIR
: Similar to DIR
, but the client is
redirected back to the modified URL (that is, a remote redirection
is used).
HTM or HTML
: Interpret as an abbreviation for an
HTML file (with extensions of .HTM or .HTML respectively).
NONE
: Interpret as is.
other_string
: The other_string is added.
For example, if NOEXT_TYPE=.GIF
,
then .GIF
will be added to these "no extension" requests.
You can use the simple-mode configurator to set the home-directoryAs a convenience, SRE-Filter will replace occurences of ~ characters in request selector with a string. Since the ~ is often used to point to a set of individual "home directories", this replacement string is stored in the HOME_DIR variable.
To select the home directory , change the value of the HOME_DIR (or HOME_DIR.HOST_NICKNAME) variable.
~JOES/PAGE.HTM
,
BRANCH/STAFF/PERSONAL/JOES/PAGE.HTM
(under your data directory, or in a virtual directory).
HOME_DIR
directory. For example,
suppose all "students" have space on the HOME_DIR
directory
of your web server; or HOME_DIR
may be on a LAN drive
accessible as a virtual directory from the web server. These
"student" directories may contain several subdirectories; one of which
is intended for WWW use, while the others are for personal
use.
The goal is to give clients direct access to the "web" related directories but not to the "personal" directories.This goal can be achieved by including a $ in the
HOME_DIR
parameter.
Specifically, the $ is replaced by the portion of the
request between the ~ and the first / following this ~
For example: If:
HOME_DIR=USERS/$/WWW
/~GERALD/RESUME.HTM
For futher details, see the discusssion of the HOME_DIR
parameter in INITFILT.DOC.
You can use the simple-mode configurator to add/remove virtual directories.By default, SRE-Filter will match the request selector to a file in the
data directory
;
or if the request is for server side processing (such as an SRE-Filter
add-on, or a CGI-BIN script), to a file in the
working directory
or CGI-BIN directory
.
While a good security feature (files not in these directories are inaccessible),
this can be an inconvenience. To remedy this inconvenience, one can
define "virtual directories" using the VIRTUAL_FILE.
Basically, SRE-Filter will compare the
starting portion of a URL to see if
it matches an entry in VIRTUAL_FILE. If it does, the
directory listed
in the entry is used (instead of the data directory or working directory).
Thus, you can make available a wide, but controllable, set of "local" directories (on or LAN accessible from) your server.
In addition, "virtual directories" can point to "remote" directories on other http servers -- SRE-Filter will attempt to retrieve the file from the remote server; without using a redirection.
To clarify: to the client, remote virtual directories are indistinguishable from "local" virtual directories.
In particular, HTML files with server side includes can be remotely located -- and if the remote server makes no attempt to process these server side includes, SRE-Filter will!
For details on the use of the virtual directories, see the description of the VIRTUAL_FILE .
Note that entries in the VIRTUAL_FILE may be "host specific"
CHECK_ALIAS should have one of the following values:
GETINFO
into: INFODIR/GETINFO.HTM
.
To use SRE-Filter's searchable index facility,
one should create an alias of the form:
INDXFILE.HTM?* DOSEARCH?FILE=FILE.NAM&TARGET=*
Where
INDXFILE.HTM
is the name of a
searchable index
(that is, an HTML document containing an <ISINDEX> element).
DOSEARCH
instructs SRE-Filter to
"execute the DOSEARCH add-on."
FILE.NAM
is the name (relative to the data directory,
or in a "local" virtual directory) of the ASCII text file to be searched.
If you want to use some other text search program, you could set up
an alias of the form:
SEARCHIT.HTM?* SRCHREXX?*
target http://new.site.org/newname
target !TEMP http://new.site.org/newname
target !MOVED http://new.site.org/newname
target !NOTIFY http://new.site.org/newname
PROJ/P48.HTM http://www.prez.edu/truman.htm
PROJ/LINCOLN/* !MOVED civ.war.org/p1/alinc.htm
CGI-BIN/MAKEMAP/?NEB+IOWA
is recieved
MAKEMAP D:\EXES\GIS
entry
Note that if this MAKEMAP D:\EXES\GIS
entry did not exist,
SRE-Filter would look for a MAKEMAP program (such as MAKEMAP.CMD) in your CGI_BIN_DIR
directory.
404 Not found
response
is returned to the client, along with a short description. You can put
a customized message in this short description. In particular, you
can put a link to your "home page".
The NOT_FOUND_URL variable
contains this customized message. By default, it's value is:
<a href="/"> Visit the HOME_NAME home page? </a>
Note that HOME_NAME
will be replaced by the current (possibly host
specific) value of the HOME_NAME variable.
Thus, the befuddled client merely needs to click on this link to get some kind
of access.
Notes:
BIG/SERVICE: Your Friendly Government Agency!
.
HOME_NAME
=" "
to send a generic message!
The simple-mode configurator can be used to .. Perhaps the greatest advantage of SRE-Filter is the extensive support for dynamic document creation. In particular, server side includes of files, text strings, and transient variables (such as the time, date, and number of requests) are easy to accomplish, requiring modifications to your HTML documents, and not to the server software.The basic idea is that when writing an HTML document, special keyphrases are inserted. When the document is requested, SRE-Filter will detect these keyphrases, and perform the desired action.
The basic structure of a keyphrase is:
<!-- KEYWORD argument
-->
where:Examples:
- KEYWORD can be one of several phrases:
Note: If desired, you can add a "preface" (such as the ! character) to the definitions of the first six keywords (i.e.; !REPLACE is used instead of REPLACE). See the description of the KEY_PREFACE variable in INITFILT.DOC for details.
- SELECT -- used to selectively include blocks of HTML code
- REPLACE -- used to "replace strings"
- INCLUDE -- used to "include files"
- OPTION -- used to "include options sent by the client"
- INTERPRET -- used to "include the results of a custom-designed REXX routine"
- CACHE -- controls SRE-Filter's server side include CACHE
- #cmd -- NCSA HTTPD style server side includes
- argument is a string containing "arguments" that are used, in conjunction with the keyphrase, to dictate how SRE-Filter should modify this document.
<-- REPLACE CREATION_DATE --> : <-- INCLUDE ADDLIST.HTM -- > <-- #FLASTMOD VIRTUAL="SAMPLES/SREFILTR.HTM" -->
Notes
- When SRE-Filter finds a keyphrase, it will ALWAYS remove the keyphrase, and replace it with the file or string dictated by the KEYWORD argument . This replacement occurs at the position in the document that was occupied by the keyphrase. If a faulty keyphrase is encountered (i.e.; a non-supported argument is used), SRE-Filter will remove it.
- The SELECT keyphrase is a little different: it dictates whether the "block of HTML code" that follows the SELECT keyphrase is to be retained. Thus, it is best thought of as a "server side exclude" -- the keyphrase is always removed, but sometimes so is a lot of other stuff!
- The CACHE keyphrase is used to control SRE-Filter's caching of documents that contain server side includes.
- The NSCA HTTPD style server side includes are provided for completeness sake; some of them are redundant with REPLACE and INCLUDE.
- You can instruct SRE-Filter to perform server side includes only on files that have a .SHT or .SHTML extension. See the description of the SSI_SHTML_ONLY and SSI_EXTENSIONS parameter (in INITFILT.DOC) for details. Or, you can use the simple-mode configurator.
Headers and Footers
In addition to server side includes using keyphrases, a multi-line header and footer can be automatically added to all HTML documents. The header is added just after the (first) <BODY > tag in the document (see the notes for an exception to this rule); and the footer is added just before the (last) </BODY > tag.To set these, just change the values of the HEADERS. (for headers) and the FOOTERS. (for footers) stem variables located in INITFILT.80. Or, you can use the simple-mode configurator.
Notes:
- Headers and Footers can contain keyphrases.
- To suppress addition of a header and/or a footer, set HEADERS.1=0 and/or FOOTERS.1=0.
- If the HEADER starts with <BODY>, then the HEADER will replace the first <BODY> element.
- The various server-side includes are only attempted for HTML files.
- If you will NEVER have server-side includes, you can set the NO_INCLUDE variable to equal "YES". This will speed up request processing a bit.
- HEADERS., FOOTERS., and NO_INCLUDE can be host specific (see INITFILT.DOC for details).
- Instead of using <!-- and --> as keyphrase delimiters, you can specify your own set(s). See the description of DELIM_1 and DELIM_2 in INITFILT.DOC for details.
Description of Keyphrases
The REPLACE keyphrase
Syntax:<!-- REPLACE Varname -->
, whereVarname
is the name of a static or dynamic variable.The REPLACE keyword is used to insert dynamically determined information and short (static) strings into a document. SRE-Filter recognizes the following "built-in"
Varnames
:A special argument can be used to extract variables from the GoServe environment, which contains many variables created by SRE-Filter.
- DATE : todays date (i.e.; 27 Aug 1988)
- TIME : current time (local) (i.e.; 4:54pm)
- DATEGMT : current date (at GMT)
- TIMEGMT : current time (GMT) -- a GMT is appended
- CREATION : Short message stating (local) creation time and date of the document
- CREATION_DATE : Creation date of document
- CREATION_TIME : Creation time of document
- SERVERNAME : The IP name of the server (i.e.; WWW.MYORG.NET). If multiple hosts, this will be the IP name of the server to whom the request was directed.
- SERVER : the server software (i.e. GOSERVE ver. 2.45 )
- WEBMASTER : The Webmaster's address **
- USERNAME : The client's name, i.e. johndoe.somewear.net (or his numeric ip address if a name is unavailable)
- HOME_NAME : The host specific colloquial (not necessarily IP) name of this domain **
- URL : The current URL (includes the servername and serverport)
- BROWSER: The name of the web browser the client is using (sometimes referred to as the 'User-Agent')
- FILTER_NAME : The name of the filter (i.e.; SRE-Filter ver 1.2).
- REFERER : The address of the server that refered the client to your site (often unavailable).
- INHOUSE.n : n = an integer. If the user has INHOUSE privileges (but does not have SUPERUSER privileges) then replace with the INHOUSE.n variable **
- SUPERUSER.n : n = an integer. If the user has SUPERUSER privileges then replace with the SUPERUSER.n variable **
- HITS and HITS_FILE: A string stating the number of hits for the "URL" or the file (respectively), being requested (the number of hits is stored in the COUNTER_FILE ).
Note that a given document may be reached via several different selectors (say, due to several aliases pointing to the same file), each of which will have a seperate counter. In contrast, each FILE has a unique counter. Also note that on multiple host servers, otherwise similar selectors (that may even point to the same file) will have seperate URL counter entries -- but (if they do point to the same file) will have only one FILE entry.- COUNTS and COUNTS_FILE: Same as HITS, but just the number is displayed.
- OPTION_COUNTS.n and OPTION_HITS.n : Normally, the same as COUNT and HITS. But, if an option has been recieved, use the value of the nth option (rather then number of hits).
- READ_HEAD : Displays (using a <PRE>> format) the "request header". This can be a handy debugging tool.
- PREFILTER_RESULT : Displays "status" message returned by the PREFILTR routine. If the PRE_FILTER variable equals NO, PREFILTER_RESULT will be an empty string.
- VARIABLE.varname will extract the varname variable, where varname is either in the GoServe/SRE-Filter environment, or a value hard coded in SREFILTR.80. For example, VARIABLE.HOST_NICKNAME refers to "host nickname" (if available) to whom this request was directed.
Last of all, if none of the above matches argument , the REPSTRGS_FILE is searched for a match (which might be host specific). If a match is found, the information from this entry in REPSTRGS_FILE is used. See REPSTRGS.IN for the details, (or you can use the simple-mode configurator).
Notes
- The **'ed variables are user-configurable "static" variables that can be modified by editing INITFILT.80. All of them may be host specific.
- To create a host-specific "static" variable, you should include a .HOST_NICKNAME. For example, for a host with a nickname of ZOO, the WEBMASTER should be specified (in INITFILT.80) using:
WEBMASTER.ZOO='MANAGER@ZOO.YOURSTATE.ORG'
.
If a matching host-specific entry does not exist, the "generic" entry (in this case, WEBMASTER) would be used.- Both "built-in" and "user-defined in the REPSTRGS.IN file" variables can be host specific (using the above syntax in both cases).
- When including static variables in your HTML documents, the .HOST_NICKNAME should not be appended!
For example, if the ZOO host is asked for INDEX.HTM, and INDEX.HTM containsthe tag: <!-- REPLACE WEBMASTER -->; SRE-Filter would look up the value of the WEBMASTER.ZOO variable.
In contrast, if the tag were: <!-- REPLACE WEBMASTER.ZOO --> SRE-Filter would look for WEBMASTER.ZOO.ZOO.
- Dynamically determined information consists of variables that are determined at the moment the request is recieved. For example:
<!--REPLACE TIME
-->
will cause the current (local) time to be placed in the document.- Static information is the same for all requests (to a given host). Inclusion of static variables can be a great convenience when writing sets of HTML documents, especially if phrases that occur in all these documents are subject to frequent change (say, a daily message).
- The HIT_CACHE_LEN and HIT_OWNER_SUPPRESS variables can be used to reduce the number of false hits.
The INCLUDE keyphrase
Syntax:<!-- INCLUDE FileName -->
, where FileName is the name of the File to be included.The INCLUDE keyphrase instructs SRE-Filter to include the contents of a file into the requested document (where argument is the file-to-be-included). For example,
INCLUDE PHONE.LST
would cause the contents of the filePHONE.LST
(in your data directory) to be inserted into the document.
Notes
- It is recommended that INCLUDE keyphrases appear on seperate lines of a file (with no other HTML statements or text).
The file is treated as a long string (that contains linefeeds) and replaces the keyphrase -- anything before or after the keyphrase (on the same line) will be placed just before and after the contents of the file.- FileName is interpreted relative to the Data Directory, or to a "local" virtual directory .
- The INCLUDE keyphrase will check for potentially fatal, recursive file INCLUDES, and disallow them. Since this check is very primitive, it will also disallow multiple INCLUDEs of the same file. If you need to include a file multiple times (say, to position a button bar at the top and bottom of your document), you can use the #INCLUDE keyphrase.
- INCLUDE, in combination with the INTERPRET keyphrase, facilitates customization of your documents , based on client specific informaton (such as the
user agent
)!
Return to table of contents
The OPTION keyphrase
Syntax:<!-- OPTION n -->
, where n is an integer.The OPTION keyphrase instructs SRE-Filter to insert the contents of the n'th OPTION into the document.
OPTIONs are contained in the portion of the request following the ? character.
How to pass OPTIONs to a document
To pass OPTIONs to a document, the selector portion of the request should have the form:
HTMLFILE.HTM?Your+OPTION+1&OPTION2
- In this example ....
OPTION 1
keyphrases in HTMLFILE.HTM would be replaced with the stringYour OPTION 1
OPTION 2
keyphrases in HTMLFILE.HTM would be replaced withOPTION2
.OPTION 3
keyphrases in HTMLFILE.HTM will be removed (since only two options appear in the request).- URL character decoding is done on the string following the ? character.
- You can have as many OPTIONs as you like, each seperated by a &.
OPTIONs and searchable indices.
When a browser responds to document containing the <ISINDEX > element, the request will have the sameHTMLFILE.HTM?word1+word2
format. In order to properly respond to such "searchable index generated" responses, you should set up an ALIASNotes
- The OPTION keyphrase is somewhat obsolete -- the use of cookies may be more appropriate (see SAMPCOOK.HTM for an example).
- OPTIONs can be useful as a simple means of keeping track of where the client came from. For example, you can use the OPTIONs keyphrase to include message about "where you came" from in linked documents.
The SAMPOPT1.SHT and SAMPOPT2.SHT files bundled with SRE-Filter demonstrates this use of OPTION (careful, it causes problems if disable SSI).Another example (albeit trickier), is to use OPTION keyphrases to customize URLS, with the goal of helping the client return to one of several potential "invoking" documents.- The OPTIONs are stored in the OPTLIST. "stem" variable, with OPTLIST.0 containing the number of OPTIONs recieved. This information can be used in INTERPRET keyphrases to help customize documents.
The INTERPRET keyphrase
Typical syntax:<!-- INTERPRET FILE xxx.RXX -->
INTERPRET provides a means of including and executing a REXX-code-block
There are two major variants of the INTERPRET keyphrase:
INTERPRET CODE statment 1 ; statement 2 ; ...
; where eachstatement n
is a valid REXX statements (and each statement is seperated by a semi-colon). For example:
INTERPRET CODE tmp1=server() ; interpret.result=tmp1
INTERPRET FILE Filename arg_list
, where:Filename
is a file containing REXX code (filename
is assumed to be relative to the working directory, or to a "local" virtual directory).arg_list
is an optional comma delimited list of "arguments"Example
then the phrase It is late, GO TO BED! will appear in your document whenever your local time is between 11:00 P.M and 6 A.M.
- If your document contains the keyphrase:
- <!-- INTERPRET FILE GOBED.RXX-->
- and GOBED.RXX contains:
- tmp1=time('h') ; if tmp1<6 | tmp1>23 then interpret.results=" <em> It is late, GO TO BED! </em> " ;
In general, the INTERPRETed REXX-code-block should include one or more of the following:
- A SAY or AUDIT command (for writing status messages)
- Assignment of a (possibly multi-line) string to the INTERPRET.RESULTS variable. The contents of INTERPRET.RESULTS is written into the document (at the location the keyphrase occupies).
- A QUEUE (or PUSH) command -- with each QUEUEd (or PUSHed) string automatically appended to INTERPRET.RESULTS (see the discussion of the USE_STDOUT variable in INITFILT.DOC for more details on this #EXEC-like feature)
- RETURN 'stuff to include'
Notes:
- When processing a REXX code-block, syntax checking is turned on -- coding errors will cause the code-block to fail, but will not crash GoServe.
INTERPRET FILE
blocks are "interpreted" by SRE-Filter -- they are not called as external procedures.Therefore: an INTERPRET FILE code-block can not contain procedures or subroutinesThis inconvenience can be partially ameliorated by calling "external" procedures from within the INTERPRET FILE code block.- If a
RETURN
is encountered in an INTERPRET file, the "returned" string is used as the server side include (the value stored in INTERPRET.RESULTS, and output from PUSH and QUEUE statements, are ignored).- The comma delimited
arg_list
can be accessed using the usual REXX syntax (i.e.;parse arg aval1,aval2
). You can include up to 10 arguments.- The following functions and variables can be accessed by a REXX code block:
host_nickname
: The host nickname (useful on multi-host servers)servername
: IP name of the server (useful on multi-host servers)host_nickname
: SRE-Filter's host nickname (for thisservername
)optlist.
: The option list stem variable (the "options" following a?
in the request).privset
: A list of the client's privilegesverbose
: The VERBOSE variable (higher values for more PMPRINTF output)docname
: The "action" portion of the requestusedfile
: The fully qualified name of the requested fileGet_value(env_var)
: TheGET_VALUE
function will return the value of an SRE-Filter "environment" variable.- You can access the GoServe functions (such as the REQFIELD function).
There are several SRE-Filter procedures, including string manipulation routines, that you may find useful.INTERPRET
will be ignored if NO_PROCESSING='YES', or if a NO_SSP permission applies to this selector.INTERPRET CODE
keyphrases will be ignored if NO_INTERPRET_CODE='YES', or if a NO_CODE permission applies to this selector.
Note that INTERPRET FILE keyphrases are not effected by NO_INTERPRET_CODE- When using
INTERPRET CODE
: each statement must end with a semi-colon.- If you wish to have INTERPRET CODE conditionally include a ssi keyphrase (say, an <!-- INCLUDE FILE --> keyphrase), you'll have to use
'<!'||'--'
and'--'||'>'
as delimiters.- If your needs are complicated, you might want to write your own SRE-Filter add-on; typically this would be a REXX routine that would generate an HTML document that SRE-Filter would then return to the client
However, when combined with the other keyphrases, the INTERPRET keyphrase can accomplish quite a bit of customization!- DIRINFO.RXX, USETABL2.RXX, and ADDPRIVS.RXX are samples of
INTERPRET FILE
files (ADDPRIVS.RXX uses thearg_list
).- Obsolete: An INTERPRET_FILE can be used to store small rexx-code-blocks using a "label" system. However, we no longer recommend this method.
The SELECT keyphrase
The SELECT keyphrase is qualitatively different from the other keyphrases -- it acts as a "conditional server side exclude" of HTML code that is already in your document. The power of SELECT is the ability to check dynamic variables (such as the client's IP address, or his user agent), and retain or exclude a portion of the HTML document accordingly.A SELECT keyphrase expects an argument containing a block of REXX code, just like the code blocks used in the INTERPRET keyphrase. This block of code will be
interpreted
. The code block should "exit" with either aRETURN 1
orRETURN 0
, where:The "subsequent HTML text block" is delimited by a RETURN 1
: the subsequent HTML text block is retainedRETURN 0
:the subsequent HTML text block is deleted.SELECT END
keyphrase, and can be of any length (if noSELECT END
keyphrase is encountered, the remainder of the file is dropped). Lastly, if there is no RETURN, the code block is retained (i.e.; the default is RETURN 1).Example:
<!--SELECT tmp.1=extract('clientaddr') ; if wordpos(tmp.1,owners)=0 then return 0; --> <em> Hello! You have SuperUser Privileges !! </em> <!-- SELECT END -->Note thatRETURN 1
is assumed by default, hence when the clients IP address is listed in the an OWNER variable (and the code block ends without a return) theHello! ...
message will not be deleted.
Notes
- If you've played around with the DELIM_1 and DELIM_2 parameters: when entering the beginning and end SELECT keyphrases, you must use the same set of delimiters.
- SELECT will not be done if:
NO_PROCESSING='YES', NO_INTERPRET_CODE='YES', or if either a NO_CODE or a NO_SSP permission applies to this selector.- SELECT blocks can not be nested. They may contain other keyphrases.
- Older versions of SRE-Filter used SELECT.RESULT=1 and
SELECT.RESULTS=0
instead ofRETURN 1
andRETURN 0
(respectively). This is still supported, but it is not recommended.- SELECT keyphrases are processed somewhat more slowly then other keyphrases -- for small replacements, you might want to use INTERPRET keyphrases instead.
The CACHE keyphrase
You can use the simple-mode configurator to enable caching of server side includesTo improve performance, SRE-Filter will "cache" HTML documents that have server side includes (SSI). This caching is of the document after the SSIs have been performed. In simple cases SRE-Filter can send this cached file, without having to repeat the actual process of SSI lookups, etc. Needless to say, this can greatly improve server performance.For a complete description of SRE-Filter's SSI-caching algorithim, and of the various options available for the CACHE keyphrase, please read SSICACHE.HTM . What follows is a very brief description of the options available for the CACHE keyphrase.
- <!-- cache no -->
- Do not cache this document
- <!-- cache asis -->
- Use a cached version "as is"
- <!-- cache notrigger -->
- Do not check the trigger files; but do check the own file.
- <!-- cache duration nnn -->
- Set the lifespan for this file's cache entry, where nnn is the number of (possibly fractional) days.
- <!-- cache trigger file1 file2 .. -->
- Check the file-stamps of these fully qualified files.
- <!-- cache trigger_type stamp -->
- Selects which field(s) of the trigger file-stamps to check.
Support for NCSA HTTPD server side include
With only minor exceptions, the server side include syntax of the NCSA HTTPD server is fully supported by SRE-Filter. These are invoked with a keyphrase of:Where<!-- #cmd option="value" -->
cmd
can be: INCLUDE CONFIG FLASTMOD FSIZE ECHO EXEC
Details on NCSA HTTPD server side includes
Need more information on NCSA HTTPD server side includes?
- #INCLUDE: Include a file.
option
can be VIRTUAL or FILE. If VIRTUAL then value should be a relative file name (relative to the data directory or to a "local" virtual directory). If FILE, then value is interpreted relative to the directory containing the "requested file" (the file that contains this server side include).Note: #INCLUDEd files will not be checked for recursive includes. To check for recursive includes, use the <!-- INCLUDE filename --> keyphrase.- CONFIG: Configure some options.
option
can be:
- ERRMSG: Error message to display when a bad request is made
- SIZEFMT : If
value=ABBREV
, will display kilobytes or megabytes (as appropriate). Otherwise, displays exact byte count (with commas added)- TIMEFMT: A set of instructions on how to display time and date. These are based on the strftime C function -- see TIMEFMT.DOC for a description of the various options.
- FLASTMOD: Modification date of a file.
Theoption
andvalue
are the same as for #INCLUDE- FSIZE: Size of a file
Theoption
andvalue
are the same as for #INCLUDE- EXEC: Execute a CGI script.
option
should equal CMD, andvalue
should be a file name (in the CGI_BIN_DIR directory) of a CGI script (typically a REXX program or an executable). Note that the EXEC server side include captures the output of the script, and includes this output in the requested document. EXEC will not be done if NO_PROCESSING=YES, or if a NO_SSP permission applies to this selector.- ECHO: Value of a variable
option
should be VAR,value
should be a variable name. This is equivalent to the REPLACE keyphrase, but with different variable names (although many of them overlap).The supported variables are:
DOCUMENT_NAME, DOCUMENT_URI, DATE_LOCAL, DATE_GMT, LAST_MODIFIED, SERVER_SOFTWARE, SERVER_NAME, GATEWAY_INTERFACE, SERVER_PROTOCOL, SERVER_PORT, REQUEST_METHOD, SCRIPT_NAME, QUERY_STRING, REMOTE_HOST, REMOTE_ADDR, AUTH_TYPE, AUTH_NAME, HTTP_ACCEPT, HTTP_REFERER, HTTP_USER-AGENTUnsupported variables are (they really aren't relevant to server side includes):PATH_INFO, PATH_TRANSLATED, REMOTE_IDENT, CONTENT_TYPE, CONTENT_LENGTH
Order of execution of KEYPHRASES
Server-side includes are processed in the following order:Note that the keyphrases in the latter part of a document are processed after freshly included keyphrases. While not the most efficient algorithim (long documents with many server side includes will cause a lot of string comparisons), it does enable recursive processing of all keyphrases
- The requested document is read into memory.
- Headers and Footers, if specified, are included.
- The document, (that now includes headers and footers), is scanned for keyphrases.
- When a keyphrase is encountered, the appropriate server side include is performed; or if a SELECT keyphrase is encountered, a server side "exclude" may be conducted.
- After each keyphrase is processed, step 4 is repeated (from the beginning of the file).
- This process continues until there are no more keyphrases.
- If you've specified more then one set of keyphrase delimiters, step 3 will be repeated for each set.
This recursive feature allows for some powerful document manipulation strictly through the use of keyphrases. In particular, the INTERPRET keyphrase can examine variables such as the "referer", the "client name", or the "OPTION list", and generate inclusions that incorporate other keyphrases (which will be processed on the next cycle of keyphrase interpretation).
There are several caveats:
- SRE-Filter will check for potentially fatal recursive includes, and disallow them. If this should occur, a simple error comment is added to the document that is sent to the client. See the description of the INCLUDE and #INCLUDE keyphrases for some important provisos.
- If you use the AUTO_HEADER feature, you should not place server side include keyphrases in the <HEAD> portion of your HTML documents.
- In general, we do not recommend use of the DO_SEND_PIECE (the send pieces as they become available) SRE-Filter option. If you do use it, note that:
- SSI-caching will be suppressed ,
- You will not be able to dynamically add cookies (say, with an INTERPRET keyphrase)
- DO_SEND_PIECE is suppressed if:
- there is more then 1 set of keyphrase delimiters, or
- the FIX_EXPIRE option is being used.
Imagemaps, CGI-Bin Scripts, and SRE-Filter add-ons
Using imagemaps
SRE-Filter provides a simple means of responding to image maps, using NCSA or CERN style "MAP" descriptor files. The key is to include a map-image indicator substring in the the "resource location" of a URL that points to a .MAP file.More specifically, when a MAPIMAGE/ substring is encountered in the action portion of the request, SRE-Filter will remove MAPIMAGE/, and assume the remaining portion of the action refers to a NSCA MAP file (similarly, a MAPCERN/ substring refers to a CERN MAP file).
For example, if the request selector is:
then the "mappable image" handler would look forMAPIMAGE/PROJECTS/BRANCH1.MAP?55,123
PROJECTS\BRANCH1.MAP
(under the data directory, or a "local" virtual directory), and use the instructions therein (in conjunction with the pixel location 55,123) to "redirect" the client.Type of regions recognized.
There are four types of regions recognized by SRE-Filter:The selected pixel will be assigned to the URL associated with the first
Circles
Rectangles
Polygons
Points
- and you should always specify a default URL
Circle
,Rectangle
,Polygon
, or 'exactly-hit'Point
it matches. If there is no such match, and there arePoints
specified in the MAP file, then the URL associated with the closestPoint
is used.However, if the distance to this closestPoint
is greater then the value of the MAX_POINTDIST variable, or if there are noPoints
specified in the MAP file, the default URL is used.Notes:
- To modify the map-image indicator substrings used to indicate a CERN and NCSA imagemap, see the description of the NCSA_ISMAP and CERN_ISMAP variables in INITFILT.DOC
- The MAP file must be in your data directory, or a "local" virtual directory (i.e.; if your data directory is E:\WWW, then the above example refers to E:\WWW\PROJECTS\BRANCH1.MAP)
- There are several handy "map" making tools available, such as MAPEDIT.
- HTACCESS is never checked for image map requests (but HTML documents that contain clickable images are subject to HTACCESS controls).
- The SAMPMAP.HTM file that comes with SRE-Filter (which uses SAMPMAP.MAP and SAMPMAP.GIF) provides a simple example.
- Your MAP file should, but need not, contain fullly specified URLS. The http:// will be added if missing; and if there is no IP address, your server's address (and port) are added.
- If the selected URL (the URL corresponding to the selected point) is a "virtual URL" (contains no IP address, and does not start with a /), then the invoking URL (minus the MAPIMAGE/) is used.
Thus,- if the selected point is associated with ANIMALS/TIGER.HTM,
- the .MAP file was /ZOO/GUIDE/CAGES.MAP,
- THEN ...
- the selected URL will be /ZOO/GUIDE/ANIMALS/TIGER.HTM.
- SRE-Filter's MAPIMAGE procedures are borrowed from Don Meyer's GoHTTP package!
Please remember that the MAPIMAGE/ "indicator string" does not refer to the MAPIMAGE/ directory!
Creating SRE-Filter add-ons
SRE-Filter provides a simple means of creating your own, custom written, SRE-Filter add-ons. The basic idea is to write an "external" REXX procedure which will do the processing, or which calls some other executable program (such as a database manager) -- and then tell SRE-Filter to call this procedure.The following steps outline how to do this:
- Create an HTML document that returns an
action
that invokes a Server Side Processing request. For example, you might include a FORM element with an ACTION attribute of "/XXX".- Create a file (that contains your REXX procedure), with the name xxx.CMD (or xxx.80) , where xxx is the action name. Note that the xxx file is relative to the working directory (or a "local" virtual directory).
For example, if the action name is /PROGS/FAST2, and \GOSERVE is your working directory, SRE-Filter will look for \GOSERVE\PROGS\FAST2.CMD.- This REXX procedure should process the request (using information passed to it by SRE-Filter), and it may call other programs (not necessarily REXX programs).
- xxx should issue GoServe completion codes (such as VAR, FILE, and STRING).
Calling Syntax
SRE-Filter will call xxx with the following arguments:
- ddir
- The data directory (typically, x:\WWW)
- TEMPFILE
- A temporary, "transaction specific" file (i.e.; D:\GOSERVE\TEMP\$341.80)
- Action
- The Action portion of the request
Example (assuming the GET HTTP method was used by the client)- If the request selector is:
/sports/getscore?city=boston&date=July+5+1994
- then the Action is
sports/getscore
- list
- If verb=GET: the partially cleaned up portion of the request selector after the ? ( + signs, tabs and linefeeds are converted to spaces, and URL decoding is performed).
In the above example, list would becity=boston&date=July 5 1995
- If verb=POST: the body of the request -- without + conversions or URL decoding.
- verb
- Either GET or POST
- Request String
- The original request string, as sent by the client to the server (includes the verb, selector, and http protocol) --without cleanup or encoding. This can be useful if & (and other special characters) may be text returned by GET method client requests
- user
- The numeric IP address of the client
- servdir
- The "base directory", where GoServe and SRE-Filter are located.
- workdir
- The fully qualified name of a "temporary, working" directory that the external procedure may wish to use (set in the TEMPFILE_DIR variable).
- privset
- The list of privileges granted to this client.
- env_id
- A substring used when specifying semaphores and queues
- transaction
- The transaction number
- verbose
- The verbose instruction (1 signifies "say intermediate results").
- servername
- The IP name of the "host" this request is addressed to. This is derived from a HOST: request header, or from the numeric ip address.
- Host_nickname
- The SRE-Filter nickname for the "host".
- Home_directory
- The "home directory" (used in ~ replacment)
Return Arguments
The xxx.CMD procedure should return a simple status message of the form:
response_code bytes_returned Description Message , file_used
Where:Examples: response_code
: the http response code (typically, 200 for GET requests and 201 for POST requests)bytes_returned
: the number of bytes transfered (not including response headers)Description message
(optional) : a descriptive message (it's transferred to the post-filter)file_used
(optional) : The program that was run (must follow a comma)200 12515 Message Succesfully Recieved, D:\GOSERVE\MESSAGES
401 302 Unauthorized access of PRIVATE DATABASE
Notes
- How does SRE-Filter recognize a Server Side Processing Request?
- If VERB=GET
... SRE-Filter looks for a ? character in the request selector , and also checks that the request is not for a text/html file.If the request selector starts with CGI-BIN/, then it's assumed to be a request for CGI-BIN processing.
- If VERB= POST
... SRE-Filter assumes it is a Server Side Processing Request.- STATUS.CMD, and STATUS.HTM, contain an example of how to request Server Side Processing using a REXX procedure.
- If a syntax error occurs, SRE-Filter will trap it and return a simple error message to the client.
- To call an external, non-REXX program from within an external REXX procedure, use the
address cmd
REXX command.- If you write HTML code to TEMPFILE (for return to the client), you should return from your external REXX procedure using:
'FILE ERASE TYPE text/html NAME ' TEMPFILE
return response_code #_bytes a message here- When using an HTML FORM to invoke an SRE-Filter add-on, we recomment that the "action" contain a leading / character (this avoids incorrect path assignment by the browser).
- If the NO_PROCESSING variable equals YES, or the NO_SSP permission is enabled, then server side processing will not be permitted.
Support for CGI-BIN scripts.
SRE-Filter supports CGI-BIN scripts (using procedures adapted from Don Meyer's GoHTTP package). To run CGI-BIN scripts, you'll need to:SRE-Filter's implementation of CGI-BIN includes a few minor modifications:
- Make sure that the CGI_BIN_DIR points to an existing directory (i.e.; \GOSERVE\CGI_BIN).
- Copy the CGI-BIN script (such as a REXX file, a binary .EXE file, etc.) to the CGI_BIN_DIR directory.
- Invoke it using a standard CGI-BIN request selector . For example:
href=http://foo.bar.net/cgi-bin/test-cgi/morepath/?a+message
.Need more information on CGI-BIN ?
- A few additional environment variables are created. The most important one is SREF_PRIVSET -- it contains a list of the client privileges.
- Requests for CGI-BIN imagemaps (such as
/CGI-BIN/IMAGEMAP/MAPFILE.MAP
) are converted into SRE-Filter IMAGEMAP calls.- SRE-Filter comes with a few sample CGI-BIN scripts (written in REXX). The TEST-CGI.CMD file (which is copied to your CGI_BIN_DIR) will display the values of the CGI-BIN variables (invoke it using /CGI-BIN/TEST-CGI?).
Using CGI-BIN scripts in other subdirectories
By default, CGI-BIN scripts are assumed to be in the CGI_BIN_DIRECTORY (say, \GOSERVE\CGI-BIN). Obviously, this can become inconvenient once you acquire a greater-than-small set of scripts. The obvious solution is to place the script files in subdirectories.Unfortunately, there does not seem to be a clearly specified method of doing this. With this in mind, SRE-Filter offers two means of specifying subdirectories containing your CGI-BIN scripts:
- Using "aliases".
This does not require changes to the standard syntax of CGI-BIN request selectors; i.e.; CGI-BIN/ASCRIPT/PATHINFO/?args ). The disadvantage is that a seperate alias is need for each script -- even for scripts in the same subdirectory.- Using the portion of the request selector before the
CGI-BIN/
The notion is to include subdirectory information before theCGI-BIN/
(the "this is a CGI-BIN script" indicator). SRE-Filter will interpret this starting portion as a subdirectory of the CGI_BIN_DIR, or as a virtual directory (in both cases,the usual HTACCESS controls may apply). This permits a powerful division of scripts into directories. The disadvantage is that it doesn't seem to be "spec" (hence may be cumbersome to port).Requests for CGI-BIN scripts are often followed by /PATHINFO/. This is typically used by the script (it does not point to the script).Example:
KIDSTORE/INVENTORY/CGI-BIN/TOYCT?toddler
This invokes the TOYCT program (with an argument oftoddler
),
which is located in (perhaps) \GOSERVE\CGI-BIN\KIDSTORE\INVENTORY\.Notes on creating CGI-BIN scripts
- When writing CGI-BIN scripts, keep in mind that a number of variables are available in the enviroment.
- You can output results with SAY statments.
- Your first SAY statements should declare the MIME type. For example:
Say "Content-type: text/html"
SayUsing PERL scripts with SRE-Filter
In order to take advantage of a broad spectrum of CGI-BIN scripts, SRE-Filter can use "non-REXX" processors to interpert CGI-BIN scripts.As long as you have the appropriate interepreter installed on your server, it's quite easy to do this. All that you need to do is tell SRE-Filter (using the INTERPRET_TYPES parameter) which interpeter to associate with different file extensions -- see INITFILT.DOC for a details on how to change the INTERPRET_TYPES parameter.
Of particular interest is the use of one of the OS/2 PERL interpreters; since there are a slew of PERL scripts out there. In fact, it's not that hard to obtain a PERL interpreter -- see PERL.DOC for details on one we've had some success with.
Special commands, and directives, recognized by SRE-Filter
SRE-Filter offers a set of special commands, and special directives for performing TCP/IP actions, computing statistics on the status of GoServe, and requesting special server actions. These command all start with an exclamation point (!), followed by a word or a phrase.The special commands are basically built-in "server side programs". In contrast, the special directives are used to modify SRE-Filter's logic.Currently available special directives
- !force/selector
- The
!FORCE
special directive instructs SRE-Filter to suppress the LOGON_FAIL_FILE feature when considering theselector
. This guarantees that unauthorized clients will be presented with a username/password screen.!FORCE
is meant to be used in the LOGON_FAIL_FILE and ACCESS_FAIL_FILE, where it provides a means by which a client can try again with a different username/password.It is also used to force re-compilation of an HTML document with server side includes (in other words, it suppresses use of the SSI-Cache).
Examples:
- !FORCE?DIR1/INDEX.HTM
- !FORCE?DIR2/INCS/INDEX.SHT
- !norecord?selector
- Instructs SRE-Filter to NOT record this request. It has the same effect as a NO_POSTFILTER permission. You can disable this option by setting the NO_NO_RECORD parameter.
- !sendas_mimetype_mimesubtype/dir/filename.ext
- The
!sendas
special action is used to force the server to send the requested file as mimetype/mimesubtype. This can be useful if you wish to create links to unusually named text, or html, files. For example, if you have a series of html documents named CHAP.1, CHAP.2, etc.; you could use links of the form:
<a href="/!sendas_text_html/mybook/chap.3">Get chapter 3 </a>Technical note:
!sendas
suppresses SRE-Filter's extension mapping method of determining the mime type (and subtype); and uses the explicitly codedmimetype
andmimesubtype
instead.
For information on customizing SRE-Filter's extension mapping, see the description of MEDIATYP.RXX.- !delsend/filename.ext
- !delsend is used to send a file from the SRE-Filter TEMPFILE_DIR "temporary file" directory. More precisely, !delsend will look for
filename.ext
in the TEMPFILE_DIR, send it (if it exists), and then delete it.!delsend is provided to facilitate writing server side programs that generate several output files -- the client would be sent back a response containing !delsend links to these several files.
Notes
- When a
!force
,!norecord
,!sendas
or!delsend
request is recieved, the !cmd portion (i.e.; the !sendas_xxx_yyy/, or the !delsend/) are immediately stripped off. Thus, SRE-Filter's various "SEL-specific" operations (such as access control, or aliasing) are performed on this "stripped" request selector .- You can end the !cmd portion with either a ? or a /. For example, the following two request selectors are equivalent:
!sendas_text_plain/hello.lst
!sendas_text_plain?hello.lst
Currently available special commands
- !authorize
- SRE-Filter will send back an "authorization" request. The primary users of this will be server administrators -- since a !AUTHORIZE link will repetitively ask for the client's username/password.
- !ping
- 'PINGS' the client.
- !statistics
- View current GoServe statistics.
- !HOST?xxx.yyy.zzz.aaa
- Get information on IP address xxx.yyy.zzz.aaa
- !reset
- Resets various GoServe statistics, and instructs SRE-Filter to re-read parameter files immediately (only available to OWNERS, or clients with SUPERUSER privileges).
- If !reset?params, then statistics will not be reset.
- !save
- Saves various GoServe statistics to the GOAUDIT.80 audit file. This is only available to SUPERUSERs, or to users with a CONTROL privilege.
- !ssi
- Used to access information on the SSI-cache. The following sub-options are recognized:
Notes: - !ssi?status : writes a (fairly technical) status report to the PMPRINTF window.
- !ssi?status=short : writes a brief status report to the PMPRINTF window
- !ssi?status=fully_qualified_file_name : writes a report on the SSI-Cache entry (if it exists) for the
fully_qualified_file_name
- !ssi?reset = Resest (clear) the SSI-cache
- !ssi?remove=fully_qualified_file_name : remove the SSI-Cache entry (if it exists) for the
fully_qualified_file_name
- !ssi?cleanup = Cleans up the SSI-Cache
- The client must have SUPERUSER privileges to invoke these !SSI options.
- After completion of the !SSI request, a short status report (in plain-text format) will be returned to the client.
- For an illustration of the use of !SSI, see SSICACHE.HTM
- !variable?var1&var2..
- Returns the value of environment variable var1, etc. (client must have SUPERUSER privileges to use this).
- !write_cache
- saves the
record_all_file
cache to therecord_all_file
- !dir?
dirname
- Invokes SRE-Filter's built in directory lister; which will create a list of all files (and subdirectories) of
dirname
.
To modify the various display parameters used by !dir, see the DIR.DOC file, or see the description of the DIR_EXCLUSION and DIR_OPTIONS parameters in INITFILT.DOC.Notes:
- The !CREATE option of the AUTO_NAME parameter also uses
!dir
to generate a directory listing.- For a more richly featured directory lister, try the GETAFILE add-on for SRE-Filter.
Common-log, browser, and referer audit files
In keeping with http server standards, SRE-Filter will maintain a common-log format audit file. Entries in common-log format audit files have the following structure:aclient.somehere.org - username [18/Mar/1996:16:24:08 -0500] "GET /SAMPLES/HELLO.HTM HTTP/1.0" 200 1251Besides offering a compact synopsis, a number of tools exist to help you analyze common-log files.
Furthermore, as an adjunct to the common-log audit file SRE-Filter can also record the browser used by each request (as well as the referer) in seperate log files.
Notes
- The WRITE_LOGS parameter is used to enable SRE-Filter's common-log, browser, and referer audit files (see INITFILT.DOC for details).
- If post-filter processing has been suppressed (with the NO_POSTFILTER permission), the request will not be recorded to these audit files.
- The SREFLOGS.INI file contains log-file control parameters. For example, you can select seperate log-files for each of several hosts, you can suppress any of these log-files, and you can suppress auditing of 400 (failure) or 300 (moved) requests.
Recording ALL requests for files
You can use the simple-mode configurator to enable the common-log audit file and the RECORD_ALL_FILE running count.As an alternative to the common-log audit file, SRE-Filter maintains a running count of requests in the RECORD_ALL_FILE . This file contains a count of requests, broken down by selector or by filename.Record refers to updating a count of the number of times a resource has been requested.
It does not mean keeping a record of each hit! If you need such detailed accounting, you should use the common-log file (or you can use GoServe's GOAUDIT.80 file).
For customized auditing needs, you can add your own procedure using the post-filter option (POSTRCRD.80 is an example of such a procedure).
To enable the RECORD_ALL_FILE, you need to set the RECORD_OPTION parameter. RECORD_OPTION can take the following values:
- NO
- Do not maintain a running count
- YES
- Record requests, but remove the the argument list from the selector (for example; remove
?hi+there
inANIMALS/MONKEY.HTM?hi+there
selector- YES_ALL
- Record requests, do not remove an argument list.
Caution: Use of YES_ALL can lead to a very large RECORD_ALL_FILE- FILE
- Record requests by fully qualified file name. In contrast, when YES or YES_ALL are specified, requests are recorded by selector
Caution: Use of FILE can lead to a very large RECORD_ALL_FILE -- careful use of wildcard entries is recommended.RECORD_ALL_FILE: Implementation details
To speed throughput, SRE-Filter caches the RECORD_ALL_FILE in memory, with a single thread responsible for updating it (the POSTFCHK thread). This raises several issues:
- The cache is written to disk on a regular basis (about once every five minutes). Thus, it is possible for the file version to be somewhat out of date; and it is possible to loose entries (say, if the server crashes).
- If you "hand edit" the RECORD_ALL_FILE (say, to add wildcard entries), SRE-Filter will read it into the cache, overwriting any changes that have been recently made. That is, any requests recieved while you were editing the file will not be recorded.
- If the cache grows large (over 1000 entries), SRE-Filter will start over -- and save the old RECORD_ALL_FILE (you won't loose information, but it will be spread over at a few files). There is one exception: SRE-Filter will retain "wild card" entries (but will reset their count to 0).
Given the above: the RECORD_ALL_FILE should be treated as an approximate (albeit probably very accurate) count of requests.To avoid loosing a few minutes of entries that will occur when you shut down GoServe, you can issue a !WRITE_CACHE request just beforehandNotes
- Wildcard matching of entries is supported, both by "selector " and by "FILE".
- By default, results are written to RECRDALL.CNT -- this can be modified by changing the RECORD_ALL_FILE variable.
- Entries in the RECORD_ALL_FILE may be "host specific". However, when FILE is specified, host information is not used.
- For YES and YES_ALL, each entry in the RECORD_ALL_FILE contains the request selector (minus the argument list if RECORD_OPTION=YES), the number of "honored" requests, and the date of the most recent request.
- For FILE, each entry contains the fully qualified file name, the number of times the file was requested, and the date of the most recent request. Note that when FILE is used, information after the ? is ignored (there is no FILE_ALL option!).
- Only "honored" requests are recorded -- requests for actions that are not honored (when ALLOW_ACCESS is enabled) are not recorded.
- If an entry matching the request selector is not found in RECORD_ALL_FILE, a new entry will be created!
- ... however, a wildcard match is always attempted. The use of "wildcard matching" is recommended if many temporary files are likely to be requested. For example, one should include a "wildcard" for the "temporary" directory in your RECORD_ALL_FILE
- The size of the RECORD_ALL_FILE cache can be increased or decreased by changing the RECORD_CACHE_LINES variable. Or, you can suppress this caching by setting RECORD_CACHE_LINES=0. See INITFILT.DOC for details.
- The differences between the RECORD_ALL_FILE method, and placing a REPLACE HITS (or REPLACE HITS_FILE) keyphrase in all documents, are:
- Nothing is written to the requested file (the client is not given info on # of hits)
- The RECORD_ALL_FILE is updated, not the COUNTER_FILE.
- All requests are tracked: including transfer of HTML documents and.GIF files, and Server Side Processing requests.
Thus, if aREPLACE HITS
keyphrase is included in a document, and RECORD_OPTION is enabled , a "count of requests" will be updated in the COUNTER_FILE and in the RECORD_ALL_FILE- For viewing the RECORD_ALL_FILE, we recommend the VURECORD SRE-Filter add-on.
Suppress recording of repetitive requests
To reduce the number of false hits from clients re-requesting a URL after a short absence (often caused by the use of a browser's "back" key), a list of current hits can be maintained. If a request selector matches a request in this list of current hits (where the request selector and the client's IP address are both used), the "count of hits" will not be augmented. Specifically, the COUNTER_FILE and the RECORD_ALL_FILE will not be augmented.In addition to "re-requests", you may also wish to not record requests from OWNERS (such as the webmaster) -- since these are likely to be due to site maintenance activity, and not due to interest in the content of the requested files.
INITFILT.DOC contains details on both of these options: see the descriptions of the HIT_CACHE_LEN, HIT_CACHE_DURATION, and HIT_OWNER_SUPPRESS variables (or you can use the simple-mode configurator).
Notes
- You can control the size and duration of the list of current hits by modifying the HIT_CACHE_LEN and HIT_CACHE_DURATION variables
- You can not record requests from OWNERS by enabling the HIT_OWNER_SUPPRESS variable.
- Similar request suppression options exists for the COUNTER.RXX hit-recorder.
Load balancing requests across several servers
SRE-Filter supports a simple form of "load balancing". When load balancing is active, the server will first check the number of active clients. If this exceeds the number specified in the LOADTHRESHOLD variable, then the client is redirected to one of the servers listed in the BackupServerList variable.For details on how to invoke load balancing, see the description of LOADTHRESHOLD in INITFILT.DOC
Creating user written pre-filter(s)
In certain cases, you may wish to have a set of custom written REXX routines examine the request before yielding control to SRE-Filter. For example, you may wish to shunt some special requests to a variety of different filters.To do this, set the PRE_FILTER variable:
You can specify the name(s) of the pre-filter procedure file(s) by setting the PREFILTER_NAME variable. By default, PREFILTER_NAME="PREFILTR" (which refers to PREFILTR.80).
- PRE_FILTER=NO: no pre-filtering will be attempted.
- PRE_FILTER=YES: a pre-filter (a REXX procedure) will be called. The pre-filter will be called after logon privileges have been determined; when individuals do not have logon privileges, the pre-filter will not be called.
- PRE_FILTER=FIRST: same as PRE_FILTER=YES, except the pre-filter is called as the very first action (before logon privileges are ascertained).
When a pre-filter is called, several arguments will be passed. The first three are generated by GoServe, the second to last is generated by SRE-Filter and is not passed when PRE_FILTER=FIRST, and the last offers a means of transfering results from earlier to later pre-filters.
parse arg source,request,sel,privs,servername,otherargs The pre-filter should return a comma delimited list:source:
myaddr port transaction who whoportrequest:
verb selector protocolsel:
action '?' awordsprivset
: list of privilegesservername
-- ip name of server (useful in multi-host environments).Other_args
-- results from prior pre-filter.
status , status message
Where status is either 0 or 1:The Status message will be stored as PREFILTER_RESULT. Note that the 0= No response to client, SRE-Filter should continue processing. 1=Response to client, SRE-Filter should exit (perhaps with response recording) REPLACE PREFILTER_RESULT
keyphrase can be used to display this status message.Notes
- If more then 1 pre-filter is called, the PREFILTER_RESULTS will be from the last pre-filter specifed in PREFILTER_NAME
- If an earlier pre-filter issues responds to the client (and issues a GoServe "completion code"), the remaining pre-filters are skipped.
Using a pre-filter to invoke the GoRemote package
The PREFILTR.80 file that comes with SRE-Filter contains a pre-filter that enables support for the GoRemote package. GoRemote (the "server remote control" package bundled with GoServe) is typically called as a special filter. By using this pre-filter, GoRemote will automatically be called (instead of SRE-Filter) when a GoRemote specific action is requested.To do this, you'll need to:
Notes:
- Make sure you have the GOREMOTE package (GOREMOTE.ZIP), and unarchive it to either the root of your data directory, or to the GOREMOTE/ subdirectory of your data directory. You'll need GOS32.GIF, GORM*.HTM and GOREMOTE.* files. Since there are about a dozen GORM* files, placing them in a subdirectory is asthetically preferable.
- Copy GOREMOTE.80 to your working directory. If GOREMOTE was installed in a GOREMOTE/ directory, copy GOREMOTE.HTM, GOS32.GIF and GORMOK.HTM to the root of your data directory
- Make sure that PREFILTR.80 (the one that comes with SRE-Filter) is also in your working directory.
- Using CONFIG2.HTM (or by direct modification of the INITFILT.80 file), set the PRE_FILTER variable to:
- YES = If you want to limit access to GoRemote:
- Clients with SUPERUSER or INHOUSE privileges will be able to view filter settings and statistics.
- Clients with SUPERUSER privileges will be able to change settings.
- FIRST = If you want all clients to have access to GoRemote
You should also set PREFILTER_NAME='PREFILTR.80'
- Set up a link to GOREMOTE (or GOREMOTE/GOREMOTE if you installed GOREMOTE in a GOREMOTE subdirectory).
- You do not need to make any other modifications, either to SRE-Filter, or to the GoRemote files!
- This pre-filter works by capturing all calls to GOREMOTE.*, GOS*.*, and GOR*.* (or to these files in the GOREMOTE/ directory). So, if you use this pre-filter, don't create files with these names (though you can create files with these names in different subdirectories).
- When requesting GOREMOTE, do not use GOREMOTE.HTM (the GoRemote filter is not expecting to see an extension).
Creating a user written Post-Filter
In certain cases, you may wish to execute some additional actions after responding to a client's request. For example, you may wish to record client specific information in a custom designed audit file. Or, you may wish to e-mail responsible parties when certain events occur (such as a file upload). Or you may wish to do both!To accomodate such needs, SRE-Filter can be told to call a series of "post-filters". As with the "pre-filter", these post-filters are custom designed REXX procedures.
To enable post-filter calling, set the POST_FILTER variable:
You can specify the names of the (possibly several) post-filter procedure files by setting the POSTFILTER_NAME variable. By default, POSTFILTER_NAME="POSTFILT".
- POST_FILTER=NO: post-filter(s) are not be called.
- POST_FILTER=YES: post-filter(s) are called.
Note: The post-filter(s) are called after a response has been sent to the client.Therefore, you can not use a post-filter to modify the response sent to the client!
To specify several post-filters, just enter a space delimted list. For examplePOSTFILTER_NAME="POSTMAIL.80 POSTRECRD.80 "
-- they will be executed in the order of appearance.
Note that if there is no extension, and no ending period, an extension of .80 is assumed (or .nnn if you are using a non-80 serverport).When a post-filter is called, several arguments will be passed:
For a simple example of a post-filter, see the POSTFILT.80 file that comes with SRE-Filter. For more useful examples, see POSTMAIL.80 (an e-mail alert routine) and POSTRCRD.80 (a client info recorder).
- amessage : a short SRE-Filter status message
- source : The server address and client address
- request: The request string; the verb, the selector , and the http protocol
- sel: The selector
- tempfile: The temporary file used by this transaction
- servername: The host to whom the request was directed (the servername)
- host_nickname: The "host nickname" corresponding to the servername (or blank, if there is no
HOSTS. variable for servername) - USED_FILE: The fully qualified file used by the request: such as the HTML file requested by the client, or the CGI script executed by the server.
- TheReferer: The value of the 'referer' request header -- if set, it contains the fully qualified URL which contained the link which invoked this request.
- Thread_cache: A pointer to a set of request specific information. This can be used by the SREF_READ_STATE macrospace procedure (note that post-filter actions can not issue GoServe commands). For details, see the description of the SAVE_STATE variable in INITFILT.DOC.
- RESPONSE_STATUS: The response code and the number of bytes returned. For example: 200 19218.
Using SREF_MAILIT to issue "event specific" alerts
The POSTMAIL.80 that comes with SRE-Filter uses the SREF_MAILIT procedure (contained in the SREFPRC1 macrospace procedure library) to e-mail "event specific" alerts through an SMTP Gateway. A different (set) of individuals can be notified for each of a set of events, where events are identified by the request selector (with wildcard matching supported), or by the requesting client's IP address. POSTMAIL.80 contains detailed instructions on how to generate these e-mail alerts.Notes:
- To use this facility, you must set the SMTP_GATEWAY variable to be the address of your SMTP server. If you do not have access to an SMTP server, you might be able to use OS/2's SENDMAIL program to set up your server machine as a SMTP server (see the OS/2 TCP/IP Command Reference for more information on SENDMAIL).
- Ambitious programmers can use SREF_MAILIT for more advanced e-mail purposes.
Uploading and deleting files
SRE-Filter provides two built-in "actions" for uploading files, one HTTP method for uploading files, and one HTTP method for deleting files. Each of these has it's own advantages and disadvantages:General Usage notes:
- GET_URL: A built-in action for uploading files from another HTTP server
- Requires that the client has ready access to an HTTP server.
- Given this access, uploading is quick and easy.
- PUT_FILE: A built-in action for uploading files from the client's own machine
- Requires that the client be using a browser that supports the
type=file
FORM attribute (such as NetScape 2.01 and above).- The PUT method.
- Requires that the browser (or other software) know how to issue a PUT method HTTP request. The DOPUT.CMD program illustrates the use of the PUT method.
- The DELETE method. This can be used to delete files from your server. As with the PUT method, it requires a browser that knows how to issue DELETE method HTTP requests.
- GET_URL.
Set up a URL with the GET_URL action, followed by the URL of the file you want to get, and the "local name". The request selector should look like:
GET_URL?url=http://a_url&file=relative_file_name
For example:
will copy
GET_URL?url=http://www.pc.com/price.lst&file=pcprice.1
price.lst
(on the server atwww.pc.com
) to a file named PCPRICE.1 (in theupload directory
).- PUT_FILE.
Create an HTML FORM with a
type=file
attribute. For example:<FORM enctype="multipart/form-data" ACTION="/put_file" METHOD="post"> Send this file <INPUT TYPE="file" name="relative_filename"> <INPUT TYPE="submit" VALUE="send file"> </FORM>- The PUT and DELETE HTTP methods are, by default, not permitted. To allow PUT or DELETE methods, you must include a PUT or a DELETE permission in the permission list of the appropriate entry (for the requested selector ) in the ACCESS_FILE.
- PUT and DELETE refer are relative to the data directory, or to a local virtual directory.
PUT_FILE and GET_URL usage Notes
- PUT_FILE and GET_URL store results in files relative to the UPLOAD_DIR directory, or to a "local" virtual directory. Note that if the target is a virtual directory, a PUT permission must exist for the specified file.
For example, if the relative file name is TODAY/OUTPUT.1, and TODAY has a virtual directory match, then access controls for TODAY/OUTPUT.1 will be examined for a PUT permission. In contrast, if TODAY has no virtual directory match, then output is assumed to be to the TODAY subdirectory of the UPLOAD_DIRECTORY -- and the PUT permission is not needed.- Files must be less then UPLOAD_MAXSIZE in kbytes size; and there must be UPLOAD_MINFREE kbytes free (in the UPLOAD_DIR directory) after the file is uploaded. If either condition can not be met, the file will not be uploaded.
- If the filename (after resolving for is directory) exists, an error message will be returned.
- If you do not specify a local file name, SRE-Filter will create a "unique" file name in your UPLOAD_DIR directory.
You can specify "partially random" filenames by including ? characters in the filename. SRE-Filter will attempt to create a unique file name by appropriate substitutions of the ? character. For example:
- if filename=ABC??.TXT,
- and ABC01.TXT exists,
- then SRE-Filter will use ABC02.TXT.
- For GET_URL: You must completely specify the URL (including the http:// portion).
- A short status report on success of the upload operation is written to the UPLOAD_LOG file.
- For PUT_FILE: Browsers that do not support the
type=file
FORM attribute will send back garbage (that will be ignored).- UPLOAD.HTM contains an example of the use of GET_URL and PUT_FILE.
Automatic generation of response headers
If desired, SRE-Filter will parse a requested document, extract all LINK and META HTTP-EQUIV elements found in the <HEAD>, and create the appropriate response headers. This is especially useful in response to HEAD requests.To do this, set the value of the AUTO_HEADER variable:
- NO : Do not create response headers
- HEAD: Create response headers when a HEAD HTTP method request is recieved for an HTML document.
- ALWAYS: Create response headers whenever a GET or HEAD request is recieved for an HTML document.
Notes
- GoServe automatically generates several response headers (such as content-length and file-creation-date).
- Server side includes (that might be in the <HEAD>) are not processed when a HEAD request is recieved. Furthermore, server side includes are not processed when response headers are created, but are processed when the <HEAD> is returned as part of the document.
Thus, if you intend to use AUTO_HEADER, we recommend not using server side includes in the <HEAD> portion of your HTML documents!- From an HTTP purist basis, the use of AUTO_HEADER="HEAD" would be discouraged, since the response headers returned by a GET and a HEAD request would be different. However, since many browsers understand that LINK and META EQUIV elements (in the <HEAD> section of an HTML document) are equivalent to response headers, this is probably not a serious drawback.
Defining new media types
SRE-Filter recognizes a list of the most frequent MIME media types (using the extension of the requested file). If you want to add to this list, you can do so by editing theMEDIATYP.RXX
file located in your GoServe directory.Additions to MEDIATYP.RXX may be new media types (say, an experimental MIME type), or they may be assignations of a non-standard extension to a common MIME type (say, .DOCUMENT to text/plain).
Notes
- MEDIATYP.RXX is checked only when the requested file has an extension that does not match one of SRE-Filter's pre-defined mime types. Therefore, it can not be used to redefine a built in mime type.
- Instructions on how to modify MEDIATYP.RXX are contained in the copy of MEDIATYP.RXX that ships with SRE-Filter.
- If you add new text/html entries, and you want these to be equivalent to SHTML entries (that is, html entries with server side includes permitted), you should edit the SSI_EXTENSIONS variable in SREFILTR.80 (see INITFILT.DOC for details).
Some useful facilites provided with SRE-Filter
Searching files using DOSEARCH
The DOSEARCH add-on for SRE-Filter is a generic search utility written in REXX. It searches one (or several) ASCII text files for "paragraphs" that satisfy a set of conditions that are specified in a "search string", and returns an HTML document containing these paragraphs.Using DOSEARCH.HTM
DOSEARCH.HTM is an HTML document that directly calls DOSEARCH -- it does not require using an ALIAS or an <ISINDEX> element. It also provides an easy means for setting a number of DOSEARCH options. If you are familiar with HTML FORMS, you can customize (and rename) DOSEARCH.HTM. In particular, to facilitate searches (say, of a specific document) many of the TYPE fields can be set to be HIDDEN.
DOSEARCH details
DOSEARCH looks for "search strings" within "paragraphs", and uses "meta commands" and "target specific" instructions to structure the search.By default, a paragraph is defined as being all text between blank lines. Alternatively, one can define paragraphs as single lines, or as delimited by any arbitrary character sequence.
A search string is comprised of "targets" There are two kinds of targets:
subwords
andphrases.
Each space delimited entry in the search string is treated as a seperatesubword
, except..
forphrases
, which are delimited by parenthesis; for example: (xx yy zz)
.... and phrases must be matched precisely.Search algorithims.
DOSEARCH has two modes: SIMPLE and LOGICAL.Simple mode with highlighting.
Two meta-commands and 4 "target specific" instructions are recognized. Meta-commands are signified by *& or *\ at the beginning of the search string. *& means "find paragraphs that match ALL targets in the search string" *\ means "find paragraphs that match NONE of the targets in the search string" If there are no meta-commands, the following target specific commands are recognized. & means "paragraphs MUST have this target" | means "accept paragraph if it has this target" Note that | is the default (assumed if no target specific command entered). \ means "paragraph must NOT have this target" % means "accept paragraph if it does NOT have this target" Summarizing: to be a "found" paragraph: Test 1a) Any (of several) | targets must be present, or 1b) All of the % targets must be missing If pass test 1a and 1b, then 2a) None of the \ can be present, and 2b) All of the & must be present If present, all & and | targets will be highlighted. Note that if there are no % targets specified, test 1b is ignored.Logical expression mode without highlighting
The user enters REXX-like logical expression using the following operators:& : AND operator | : OR operator \ : NOT operator @ : XOR operator ( ) : to group expressions. A sequence of words without any operators is treated as a phrase -- to treat each word as a seperate subword, put ( ) around each one. Basically, when using this mode, be liberal in your use of ( ).DOSEARCH Options
DOSEARCH supports a number of options. These options are easily set in DOSEARH.HTM, or by modifying the ALIASE(es) that invoke DOSEARCH.Technically speaking, the options appear in an option list, with each option seperated by an & character.....
- if the GET method was used, the option list is placed after a ? character in the request selector .
- If the POST methed was used, the option list is placed in the body of the request.
The option list should have the structure:
option_name=option_value&option_Nam2=option_value2&...The DOSEARCH options are:
DELIM : The paragraph delimiter. " " or 0 = blank lines (the default) "$" = Each line is a paragraph other = User specified delimiter LINE : Maximum number of lines to display of each "found" paragraph. If 0, no lines displayed (a summary will be displayed) Default is to display all lines. NUM : YES=Display the line or paragraph number, NO=Don't (default=YES) BAR : YES= Seperate each paragraph/line by a horizontal bar, NO=Don't (default=YES) EXPERT: YES= Use "logical expression mode", NO=Use simple mode (Default=NO) FILE : FILE=filename A file to search (either relative to the data directory, or in a "local" virtual directory). You can include as many FILE options as desired (each entry will be searched in turn). Furthermore, * and ? wildcards can be used. SEARCH: The search string CASE : If YES, then search is case sensitive (default is NO)Examples
Simple mode examples
( The best car)
- Finds paragraphs containing the phrase
the best car
car truck motorcycle
- Finds paragraphs that contain one (or more) of
car, truck
ormotorcycle
dog cat & store (pet pig)
- Finds paragraphs that contain one (or more) of
dog, cat
or the phrasepet pig
; and that also containstore
*& computer price memory
- Finds paragraphs that contain
computer, price
, andmemory
(must have all of them, but can be in any order)Logical mode examples
The best car
- Finds paragraphs containing the phrase
the best car
car | truck | motorcycle
- Finds paragraphs that contain one (or more) of
car, truck
ormotorcycle
( (dog & cat) | (pet pig) ) & stores
- Finds paragraphs that contain
stores
, and that contain either bothdog
andcat
, or the phrasepet pig
The SENDFILE facility
SENDFILE is an SRE-Filter facility that transfers files to a client. It can record the number of times this file has been completely sent.SENDFILE is invoked with a request selector of:
SENDFILE?selector&forcetext=1&COUNTER=option_list
(the &forcetext=1 and &COUNTER=option_list are optional)SENDFILE will ...
- Check for the existence of a file corresponding to the
selector
(the file that the selector maps to is assumed to be relative to the data directory or relative to a "local" virtual directory).- Look in the SENDFILE_FILE for an occurence of the
selector
. If none exists, create an entry.- If the
selector
exists (and the entire file was sent), the "count of requests" for thisselector
is augmented. That is, if the client killed the connection half way through, the count will not be augmented.- If a COUNTER=... options is present, COUNTER.RXX will be used (with the option_list).
- If SAVE_STATE=1 the
number of bytes sent
and thesize of the file
are stored in a SENDFILE_STATUS variable in the "thread specific" cache.- A successful transfer will have the "number of bytes sent" equal to the "size of the file".
- SAVE_STATE is set in SREFILTR.80 -- see INITFILT.DOC for details.
Notes
- If the client breaks the connection before the a file is completely sent, the SENDFILE_FILE is not modified (the request is not recorded). However, if SAVE_STATE=1, the SENDFILE_STATUS variable will be written.
- By default, SENDFILE_FILE is SENDFILE.CNT.
- When ACCESS_ALLOW is enabled, the ACCESS_FILE should contain an entry of the form
SENDFILE?*
- The SENDFILE_FILE supports wildcard matching.
- For details on the COUNTER= option_list, see COUNTER.DOC .
- If
FORCETEXT=YES
, SENDFILE will assume that all files are of "mime-type text/plain". This can be a useful trick (it tricks the client into displaying everything as a text file), but if .GIF (or other non-text files) are chosen, garbage will be displayed ( use this option with discretion)!- Entries in the SENDFILE_FILE may be "host specific".
Using Message Boxes
SRE Filter's built-in message box facility provides a mechanism for storing messages from clients. SRE-Filter provides a few simple built-in tools for manipulating these messages boxes. These include:Of perhaps greater interest, the FORUM, READMAIL, and MAILIST SRE-Filter add-ons make extensive use of these SRE-Filter messages boxes.
- A message writer. The message writer is invoked with a request selector of:
MESSAGE?messbox=messagebox&field1=value&..&fieldn=value
- Two simple facilities (VIEWMESS and ASKMESSBOX) are provided for viewing an entire message box.
- Message box access can be controlled through the use of message box privileges.
The Message Writer
The message writer is usually invoked with an HTML FORM with ACTION="/MESSAGE" (see SAMPMBOX.HTM for an example). In addition to an INPUT element identifying the "message box", you can specify a number of fields (the field name, and it's value, are written to the message box using an e-mail type format). Several of these fields are treated specially, as explained below:
- MESSBOX (recommended)
- The particular message-box to put the note into. If a MESSBOX field is not included, the note will be put into the ANYONE message-box. If MESSBOX names a non-existent message-box, a new message-box will be create.
- NAME and SUBJECT (recommended)
- These are expected, if not included a dummy value will be created. Note that each message is automatically given a TIME and DATE stamp.
- FAX, E-MAIL, ADDRESS, and PHONE (optional).
- The name, and value of these fields are written to the message box. Note that the E-MAIL field is explicitly used by SRE-Filter's FORUM add-on.
- PASSWORD and DURATION
- The PASSWORD and DURATION fields are used by the FORUM add-on to control "delete privileges" for a given message. PASSWORD will be encrypted before being written to the message box file. Note that messages boxes are not designed for security (the encryption is weak), so don't use them when confidentiality and integrity are important.
- INDEX
- Including an
INDEX=YES
field will cause MESSAGE to update an index file (with an .IDX extension). These .IDX files are used by the FORUM add-on to speed up response. Note that this .IDX file creation will occur after responding to the client (it's a post-filter type of action) -- the client will not notice any adverse impact from inclusion of anINDEX=YES
field (but other clients may be impacted by the extra workload!)- BROADCAST_LIST
- If included, SRE-Filter will simultaneously e-mail (through the SMTP gateway you set with the SMTP_GATEWAY variable) copies of your message to a list of e-mail addresses. This list, a file with name given by BROADCAST_LIST, must be in (or under)the MESSBOX_DIR. Furthermore, the client must have SUPERUSER or BROADCAST privileges, or a privilege equal to the value of BROADCAST_LIST. If not, SRE-Filter will send authorization request back to the client.
- Example: if BROADCAST_LIST=CLASS10.LST, then the client must have a CLASS10.LST client privilege.
Note that the BROADCAST_LIST file (i.e.; CLASS10.LST) should contain valid e-mail addresses, one per line.- BROADCAST_ONLY=YES
- Do not write to message box, just BROADCAST
- BROADCAST_NAME=My_name
- Used in the "from" field of the e-mail message.
- WRITE_TO_FILE=YES
- If included, SRE-Filter will write an "E-mail" like file (containing From, To, Date, and Subject fields; followed by the Contents) to your MAILBOX_DIR directory. These files are designed to be read, and processed as an "e-mail message to a list-server", by the MAILLIST (and READMAIL) SRE-Filter add-ons.
If you are broadcasting to many recipients, the use of WRITE_TO_FILE in conjunction with the MAILLIST add-on, rather then the BROADCAST_LIST, is recommended..- Other names (optional).
- The name, and value, of these fields will be written to the message box.
The difference between "other names" and the PHONE, ADDRESS, and FAX "optional" fields is merely in the order of appearance.- RESPONSE_FILE
- The RESPONSE_FILE field is used to indicate a file contaning a "response" to a posted message. If no RESPONSE_FILE is included, a very simple generic response will be returned.
- MESSAGE (recommended)..
- The (multi-line) message. It's written last of all.
Viewing Message boxes with VIEWMESS and ASKMESSBOX
The direct means of reading a message box is to simply view it using the OS/2 TYPE command (message boxes are ASCII files, with a .LOG extension, located in the MESSBOX_DIR directory). SRE-Filter also provides a simple mechanism for remotely viewing these message boxes.A request selector ofSRE-Filter also provides a simple routine that generates an HTML document that displays a list of all available message-boxes, and let's the client choose which one to view. This routine can be invoked with a request selector ofVIEWMESS?messbox=message_box
will cause SRE-Filter to transfer the message-box-file to the client.
askmessbox?An+optional+header
The "optional header" is displayed as an <H2> at the top of the document.MESSBOX Privileges
To access a message box with MESSAGE (or with the FORUM add-on), the client must have "message box specific" privileges. These are specifed by including (in the client's privilege list), entries of the form:
MESSBOX=a_message_box.
For example:MESSBOX=FORUM1
would give (read/write) access to theFORUM1
message box. Note that this is in addition to any "selector and FILE specific" access controls you may have imposed.The
a_message_box
portion of the privilege can contain the * "wildcard" character. For example,MESSBOX=FORUM*
would provide access to FORUM1, FORUM2, etc.On installation, SRE-Filter includes
MESSBOX=*
in the PUBLIC_PRIVS parameter (the effect of which is to allow open access all message boxes). You'll need to remove this from PUBLIC_PRIVS if you want to limit access to message boxes!Notes
- You can also limit access to message boxes by creating an entry in the ACCESS_FILE of the form:
MESSAGE* resource_privilege_list
(you may also want to create similar entries for VIEWMESS and ASKMESSBOX)
The SRE-Filter procedure library
SRE-Filter uses a procedure library (that is loaded into macrospace) to provide many functions. Several of these routines may be quite useful to the ambitious REXX programmer. These procedures include:A complete description of these routines can be found by downloading SREFPRC1.DOC.. For serious programmers, the source code is also available.
- SREF_MEDIATYPE: Return the MIME type of a file (based on it's extension)
- SREF_REPLACESTRG : Replace substrings
- SREF_MULTI_SEND: Expedites the creation of multi-part documents
- SREF_GRAB_FILE : Wait for file to become unlocked, and then read it into a variable.
- SREF_FIX_URL : Given a local selector , and servername, and a serverport, return a legitimate URL (including the http://).
- SREF_WILDCARD : Perform a wildcard substitution.
- SREF_JULDATE: Return the Julian date and time (days since 0/0/0, time in fraction)
- SREF_MAILIT : Send an E-mail message.
- SREF_CGIVARS : Determines one of the "CGI-BIN" variables.
- SREF_BROWSER_INFO: Check on the capabilities of the client's browser.
SRE-Filter Add-ons
SRE-Filter has a number of add-ons avaiable. You can download them from the http://rpbbcam.econ.ag.gov/srefilter site.Currently, the add-ons include:
- FORUM: A news-group and list-server discussion-group package.
The FORUM add-on consists of three components:
- FORUM: The FORUM component provides a simple, but fairly complete, multi-threaded "news group" like capability. The basic output of FORUM is a list of "links" to messages in the message box. The list is organized hierarchically, with indentation used to indicate discussion threads.
Clinking on a link causes just that message (and it's various "fields") to be displayed. The client then has a number of options: including sending a response to the message poster (given that an e-mail address is available), posting a new message to the message box (either as a response to the message, or a as new entry in the discussion thread), or viewing the next message or the next thread.
- READMAIL: The READMAIL component that will transfer mail to SRE-Filter message boxes. This mail is read from the incoming mail directory of an SMTP-gateway running on your server; for example, the SMTP-gateway provided by OS/2's SENDMAIL daemon. By combining READMAIL, and the BROADCAST_LIST option of MESSAGE, a simple "list server with automatic archiving" can be implemented.
- MAILIST: MAILIST is nearly identical to READMAIL, but is designed to run independently of SRE-Filter. MAILLIST requires somewhat more configuration, but once running does not suffer from "timing out" problems due to GoServe limitations on connect time. It also automatically checks for new mail on a timed basis; in comparision, you have to explicitily invoke READMAIL (say, by using READMAIL.HTM).
- BBS: A full featured "web based bulletin board system"
BBS is a WWW bulletin board system. BBS features include:
- Individual file descriptions can be displayed, along with file size and creation date.
- Directory specific inclusion files (i.e.; FILES.BBS) and exclusion files can be used to control file and sub-directory display.
- Optional "automatic descriptions" can be generated for HTML, .ZIP, and text files.
- .ZIP files can be "opened and displayed", with subsequent retrieval of a specific file from within the .ZIP archive
- Netscape 2.01, and other HTML 3.0 compliant browsers, can upload files by using a FORM containing a file="type" element.
- Uploads and downloads can be recorded on a per user basis.
- A flexible access control mechanism allows customizaton of which file areas a user can peruse.
- Downloads can be disallowed if a user's "download to upload" ratio grows too large.
- Directory specific headers, footers, and "exclusion lists" can be specified.
- Optional caching of directory-listings can greatly speed up throughput.
- SRCHINDX: A front-end (+ more) to the freeware SWISH search engine. SWISH is used to index the many directories comprising your web-site. It creates a compact index that it can search quickly for matching words.
SRCHINDX, the SRE-Filter front-end, enables you to search this from an HTML form. In addition, SRCHINDX will automatically create a description. You can also use SRCHINDX to create a "description index", which you can then search for matches.
- GETAFILE: A directory displayer, with user-settable display features
GETAFILE is a similar to SRE-Filter's !DIR dynamic directory display facility. It is somewhat easier to configure, and has a few extra bells and whistles.
- CALC: A character-mode scientific calculator with a web interface
CALC is a WWW-accessible scientific calculator. It uses a keyboard interface, and is based on an aging DOS program. On the other hand, it's quite feature rich: multiple equations and temporary variables can be specified, trigonometric functions are fully supported, analytic derivatives can be taken, and more...
User-configurable SRE-Filter files.
SRE-Filter uses several user-configurable files. These files contain logon information, access and accounting information, and instructions to assist in conducting server side includes. In most cases, SRE-Filter's simple mode configurator (CONFIG0.HTM) or intermediate mode configurator (CONFIG2.HTM) can be used to modify these files. In some cases, modification of these files requires the user to edit them with a text editor. This section describes these files; you will also find additional description in the "default" files that are bundled with SRE-Filter. x
Initialization Parameters File (INITFILT.80)
SRE-Filter contains a number of parameters that can, and should, be changed to fit the needs of the user. To do this, you'll need to change the the INITFILT.80 (or INITFILT.nnn, if you are using a port nnn).Changes to the INITFILT.80 can be accomplished in several ways:
- You can edit it with your favorite text editor. You might want to examine INITFILT.DOC for a detailed discussion of the meaning, and possible values, of each variable included in INITFILT.80.
- You can use SRE-Filter's simple or intermediate mode configurator. To run either of these configurators, point your browser at http://your.server/configur.htm and follow the instructions. The simple mode configurator allows you to change the most important parameters, while the intermediate mode let's you change almost all the parameters.
If you need to modify the few parameters that the intermediate mode (purposefully) ignores, you'll have to hand edit the INITFILT_FILE (and read INITFILT.DOC first!).
You must be an OWNER, or have SUPERUSER privileges, to access these configurators. Extremely cautious administrators may wish remove CONFIG2.CMD and SREFCONF.CMD, as well as all the *CFG.CMD files, from their GoServe working directory.
- A few of the parameters in the INITFILT.80 file are:
- List of supported hosts (IP addresses and IP aliases)
- Name of the default realm
- Numeric IP addresses of "owners"
- Numeric IP addresses of "in-house" clients
- Names of various SRE-Filter user-configurable parameter files (such as the USER_FILE and the COUNTER_FILE).
- Name of default (home) page.
- Messages to include as "header" or "footer" for all requested HTML documents.
Replacement strings file (REPSTRGS_FILE)
The replacement strings file contains "custom" replacement strings that can be used by the REPLACE keyphrase.Each line of this file (except lines beginning with a semi-colon, which are treated as comments) has two fields:
- A variable_name . The variable_name can be any length, but must have NO embedded spaces.
- A
replacement string
. Thereplacement string
is the string to be used when a REPLACE keyphrase with an argument of variable_name is found.Example
CONTACTContact me soon!
CONTACT is the variable_name , it may contain a .HOST_IDENTIFIER. Contact me soon!
is thereplacement string
When a <!-- REPLACE CONTACT -- > keyphrase is found, it is replaced by Contact me soon!
Note on multi-line replacements
The REPLACE keyphrase can be used for multi-line replacements (note that for replacements of more then a few lines it is probably wiser to use the INCLUDE keyphrase).To specify a multi line replacement, just enter multiple occurences of the same label -- the values will be appended. Example:
would generate 3 lines (when a REPLACE MYADDRESS keyphrase is encountered). MYADDRESS <address> My address is:</address> MYADDRESS 1301 New York Ave.<br> MYADDRESS Washington, DC <br> Notes
- The replacement strings file is REPSTRGS.IN. You can change this by modifying the REPSTRGS_FILE variable in INITFILT.80.
- REPSTRGS_FILE is only checked if the variable_name does not match one of the "built in" variables.
- Replacement strings can contain HTML elements.
- The variable_name can contain any printable character (including periods and commas).
- To specify a host specific entry, include a .Host_nickname (i.e.; VAR1.HOST1).
The HITS and COUNTS counter file (COUNTER_FILE)
The counter file is used to record the number of requests (also known as the number of hits) for this document. It is used whenever a REPLACE HITS or a REPLACE COUNTS keyphrase is processed.The counter file has syntax:
HOST_NICKNAME//AURL CTS MESSAGE_1 , MESSAGE_2
When a HOST_NICKNAME is optional. AURL
is a selector corresponding to an HTML documentCTS
is the current count of the number of requests.Message1
andmessage2
are used by HITS to display the # of requests.REPLACE HITS
keyphrase is encountered, SRE-Filter looks in the counter file for anaurl
that matches the request selector If it finds a match, theCTS
,MESSAGE_1
andMESSAGE_2
values are extracted.CTS
is then augmented by one and written back to counter file.Lastly, a "count of hits" string is included in the the requested document. This string will have the form:
Message_1 cts+1 Message_2
a generic message_1 is used if message_1 and message_2 are missingFor
REPLACE COUNTS
, the same procedure is followed, exceptmessage_1
andmessage_2
are ignored.If the request selector does not match any of theaurl
's, a new entry will be created (withCTS
=1 andmessage_1
andmessage_2
left blank).Examples
INDEX.HTM 66 You are visitor # , since <!--replace creation_date -->} PROJECTS/PROJ1.HTM 19 CALC/CALC.HTM 9 OVERVIEW.HTM 13 BRIDGES//VERMONT/COVERED.HTM 15Note the use of the message_1 and message_2 blocks in the entry for INDEX.HTM. Also note the use of a REPLACE keyphrase in the message_2 block.Notes
- Only the portion of the request selector before a ? is used.
- The default name of the counter file is COUNTER.CNT. You can modify this by changing the COUNTER_FILE.
- Wildcard matching is not attempted.
- An alternative means of recording "number of requests" is provided by setting the RECORD_OPTION variable.
- Another alternative is the SENDFILE facility, which keeps track of explicitily requested files.
- Or, you can use COUNTER.RXX -- invoked using an INTERPRET keyphrase (see COUNTER.DOC for details).
- Yet another alternative is to use external programs to generate a "counter GIF". Many of these programs will also keep track of the number of hits. For example, XCOUNT.CMD (a CGI-BIN script) will generate a simple "odometer" style counter, and also keep track of the number of hits (note that this doesn't work if the client has her browser set to not load graphics).
- In the COUNTER_FILE (and in the ACCESS_FILE, the SENDFILE_FILE, and the RECORD_ALL_FILE) ...
- The request selector is used to record entries -- before ALIAS and other replacements.
- / characters are converted to \ characters; and leading \ (or /) characters are dropped.
- lines beginning with a ; are comments.
- The "host nickname", if present, is appended to the beginning of the selector . For example, the BRIDGES//VERMONT/COVERED.HTM example signifies a selector of VERMONT/COVERED.HTM to a host with a host nickname of BRIDGES.
The RECORD_ALL_FILE counter file
The RECORD_ALL_FILE (with default name RECRDALL.CNT) is used to record all actions, such as transfers of files and requests for server side processing. It is used when RECORD_OPTION is enabled.The structure of the RECORD_ALL_FILE is similar to the COUNTER_FILE, with a few exceptions:
All requests are tracked (including requests for HTML documents, requests for .GIF files, and Server Side Processing requests). Instead of a message_1 , message_2
following theaurl cts
, the date of the last successful request for this selector written.RECORD_ALL_FILE may contain "wildcard matches" -- typically subdirectory entries. Example
PROJECT/PROJ1.HTM 9 03/01/96 TEMP/ZFVL5615.LST 1 02/28/96 DOSEARCH 4 2/29/96 OVERVIEW.HTM 7 03/01/96 TEMP/* 3 03/01/96 ZOO// SCHEDULE.HTM 120 05/03/96Note the TEMP/* entry is used as an wildcard match; if RECORD_OPTION="YES" or "YES_ALL", then all requests for files in the "temporary directory" will be recorded in the TEMP/* entry (the assumption is that one is typically not interested in specific information on temporary files). Also note TEMP/ZFVL5615.LST -- it was written when a file in the "temporary" directory was requested, and before the TEMP/* entry was added to the RECORD_ALL_FILE (the server administator must have added the TEMP/* entry on 2/29/96!).Also, note that the date is the date of the most recent "allowed" request.
Notes
- The RECORD_ALL_FILE is meant strictly for recording information -- it does not contain message strings, nor does it contain "access control" information.
- Entries in the RECORD_ALL_FILE may be host specific
- If more then one "wildcard" entry matches the requested selector , the "best match" is used.
The SENDFILE_FILE counter file
The SENDFILE_FILE counter file (with default name SENDFILE.CNT) is used by the SENDFILE facility to record transferals.The structure of the SENDFILE_FILE is:
URL_NAME #_of_times_sent date_of_last_transfer
Example
CTEMP/FOO.HTM 4 07/06/96 GOS/GO1.HTM 6 07/06/96 EAA.HTM 20 07/06/96 /SAMPLES/SRCHINDX.DOC 2 07/07/96 /SAMPLES/TESTTABL.HTM 1 07/06/96 ZOO// TIGERS.DOC 5 06/07/96Notes:
- If a SENDFILE_FILE does not exist, it will be created.
- Entries in the SENDFILE_FILE may be host specific.
- If more then one "wildcard" entry matches the request selector , the "best match" is used.
The ACCESS_FILE access control file
The ACCESS_FILE (with default name ALL_FILE.CTL) contains access information on a SEL-specific basis. Each entry in the ACCESS_FILE can contain 5 types of information:
- A selector . This may contain the wildcard (*) character (required)
It also may be preceded by the host nickname.- A space delimited
resource privileges
list (optional).- A space delimited
permissions list
(optional)- A
SEL-speciflc realm name
(optional)- and a
SEL-specific access failure file
(optional)The structure of each entry in the ACCESS_FILE is:
host_nickname// SEL resource_privilege_list , permissions_list , realm_name , failure_file
Note that, if present, thepermissions_list
must follow a comma, therealm_name
should follow a (second) comma, and thefailure file
should follow a (third) comma.
Special Entries: REALM-specific, additional resource privileges
Each realm can have associated with it a list of realm-specific, additional resource privileges. If available, this (space delimited) list is added to theresource_privilege_list
.
The syntax of these special entries is:
!REALMrealm_name realm_privilege_list
Note that if there is no !REALM entry for a given SEL-specific realm, then just the resource_privileges list is used.
Also note that an empty resource_privileges list implies "all clients are allowed access", but only if there is no realm-specific, additional resource privileges for this SEL.
Examples
CLIENT1/PROJ1.DOC VENUS , NO_SSI , PLANET_2 BATMAN// CLIENT1/* ROBIN GOHTAM , NO_CODE , REALM_BAT , D:\GOSERVE\NOBATS.HTM WORKAREA/MAPLE.IN INHOUSE MERCURY WORKAREA/SYCAMORE.IN MERCURY VENUS MARS &EARTH USAGE.TXT NO WEATHER* INHOUSE , , CLIMATE_GROUP READ.ME * VIEWMESS?messbox=SECRET2* INHOUSE PUBLIC/* * GUEST/* * , NO_SSI NO_SSP UPLOADS/* * , PUT DELETE OREGON/* , , REALM1 imgs/*.gif * , cache no_virtual no_alias no_postfilter no_htaccess !REALM REALM1 BIGFOOT SASQUATCH ZOO// EMPLOYEES.HTM MANAGER
- CLIENT1/PROJ1.DOC (the file that this SEL maps to) will only be sent to clients having VENUS privileges. Furthermore, server side includes will not be done on this file. Lastly, should a client not have a VENUS privilege, the query for a new username/password will display the PLANET_2 realm name.
- BATMAN// CLIENT/ is valid only for requests to a host identified with a host nickname of BATMAN. If the client does not have a ROBIN or GOTHAM client privilege, SRE-Filter will return the D:\GOSERVE\NOBATS.HTM access-failure file. Also note that
SELECT
andINTERPRET CODE
server-side-include keyphrases are not permitted in CLIENT1/ documents.- WORKAREA\MAPLE.IN will only be sent to clients having either an INHOUSE or MERCURY privilege. Note there is no SEL specific realm, so queries for a new username/password will always display the default realm (as specified in the THE_REALM variable.
- WORKAREA\SYCAMORE.IN will only be sent clients having "one of" the MERCURY,VENUS, or MARS privileges; and who also "must have" an EARTH privilege.
- USAGE.TXT is never sent (except to SUPERUSERs)
- All selectors that start with
WEATHER
are available to users with INHOUSE privileges. If necessary, authorization requests will display the CLIMATE_GROUP realm name.- READ.ME is always sent
- The SECRET2 (or SECRET2x) message box can only be examined by clients with an INHOUSE privilege.
- All files in the PUBLIC directory are always sent.
- All files in the GUEST directory are sent, but no server side includes (or server side processing) is attempted for these files.
- Selectors pointing to the UPLOADS directory can use the PUT and DELETE HTTP methods (to upload and delete files).
- All files in IMGS\ that end with .GIF will be sent to everyone, will be cached, aliasing won't be done, virtual directory lookup won't be done, directory-specific (HTACCESS-file) access controls will not be attempted, and recording and "post-filtering" (including common-log audit file recording) will not occur.
- All files in OREGON/ will be sent to individuals with one of the privileges in the REALM1 list of "additional resource privileges". The !REALM entry defines these to be BIGFOOT and SASQUATCH.
- For the ZOO host, access to EMPLOYEES.HTM is limited to clients with MANAGER privileges.
ACCESS_FILE summary
- Assuming that ALLOW_ACCESS is enabled...
- if a matching entry (in the ACCESS_FILE) contains no resource_privileges list, and there is no SEL-specific realm (or no !REALM entry for a SEL-specific realm), the selector is honored (the appropriate file is sent to the client)
- if there is no matching entry, either explicitily or as a wildcard, the selector is not honored.
- If the ACCESS_FILE does not exist, the selector is not honored.
- If the resource_privileges list contains
NO
,the selector is not honored- If the resource_privileges list contains
*
,the selector is honored- If the client has been assigned any of the ONE_OF resource privileges, and all of the MUST_HAVE privileges, in both the
SEL-specific resource_privileges list
and therealm-specific, additional resource_privileges list;
the request will be honored.- Currently, ten permissions are recognized: NO_SSI, NO_CODE, NO_SSP, CACHE, PUT, DELETE, NO_HTACCESS, NO_VIRTUAL, NO_ALIAS, and NO_POSTFILTER.
- If the permissions_list is missing, server side includes and server side processing will be permitted, caching may be suppressed (if ALLOW_ACCESS or CHECKLOG is enabled), requests using the DELETE and PUT methods will not be honored, directory-specific (HTACCESS-file) access controls will not be suppressed, aliasing may occur, virtual directory lookup may occur, and post-filtering and file recording may occur.
- The suppression of server side processing on a per-selector basis is tricky: consider that requests for a document with an HTML
FORM
is not the same selector invoked by this FORM's "action". Use it with care!- Caching can greatly speed up response time (especially of in-line image rich documents), but it also can completely subvert all access controls (including logon requirements). Use it with care!
- ACCESS_FILE entires may be host specific.
- If there are multiple wildcard matches, the "best" match is used.
The virtual directory file
The VIRTUAL_FILE (with default name VIRTUAL.IN) is used to specify local or remote virtual directories.Each line of the VIRTUAL_FILE contains:
HOST_NICKNAME SEL_fragment TARGET limitation_list
.
- The optional
host_nickname
dicates which host the entry applies to.
- If the
limitation_list
is not enabled, or if the request is one of the types listed in thelimitation_list
, then theSEL_fragment
is compared to the beginning of the request selector.If it matches, the
TARGET
directory, along with the portion of the request selector that follows thisSEL_fragment
is used as the root directory for this request.
Note that there are two kinds of virtual directories: local and remote.Examples:
- LOCAL
- Local directories are located on, or are LAN accessible from, the server.
Local directories should be specified using a fully qualified directory name.- REMOTE
- Remote directories are "web" accessible from the server. That is, requests to other http servers can be used to obtain files in these directories (assuming the other http server allows unrestricted access to the desired URLS).
Remote directories should be specified using a complete URL, that must start with http://.
Remote directories are only used for document requests.
LOCAL/DISK1/PROJECTS d:\work\PROJECTS*
- A selector of
/LOCAL/DISK1/PROJECTS/BOB/PLANE.HTM
is "mapped" toD:\WORK\PROJECTS\BOB\PLANE.HTM
REMOTE1/ http://www.their.org/
- If a selector of
REMOTE1/TURNIPS.HTM
is recived, then SRE-Filter will attempt to obtainREMOTE1/TURNIPS.HTM
from the http server atwww.their.org
. This retrieval does not require a "redirection" by the client -- all the work is done by SRE-Filter!PHYSICS// STRINGS/ d:\theory\basic\strings !HTML
- Request to a host with a host nickname of PHYSICS, that begin with STRINGS/, are mapped to files in
d:\theory\basic\strings
. However, only document requests will use this entry -- cgi-bin, SRE-Filter add-on, and file upload requests will not use this entry.ALASKA/ r:\states\alaska
- A request for
ALASKA/BEARS.HTM
"maps" toR:\STATES\ALASKA\BEARS.HTM
.If R: is a LAN drive, and the server is not logged on, this could cause trouble!Notes
Virtual directories are used in the following circumstances:
- Resolving requests for files (such as HTML documents)
- Including files with the <!-- INCLUDE filename.ext --> server side include keyphrase
- Determining the location of an imagemap file
- Determining the location of a CGI-Bin script
- Determining the location of an SRE-Filter add-on
- Determining where to PUT an uploaded file
- Entires in the virtual directories file may be "host specific".
- The * after a (remote or local) directory name means "allow access to subdirectories". Thus, if the * had been missing, BOB/PLANE.HTM would not have been honored (a "no access allowed" error would be generated);
Similarly, a request forREMOTE1/FRUITS/APPLE.HTM
will not be honored (since there is no * at the end ofhttp://www.their.org/
).- The
limitation_list
is used to limit the applicability of a virtual directory entry. For example, you may wish to have some virtual directories apply only to document requests, and some only to upload requests.The
limitation_list
(if present) should contain any mixture of the following special keywords:Note that you can contain more then one entry in the space delimited - !HTML --- use this entry for document requests
- !CGI-BIN --- use this entry for cgi-bin requests
- !ADDONS --- use this entry for SRE-Filter add-ons
- !UPLOAD --- use this entry for file uploads
limitation_list
. Including all four entries is the same as not having alimitation_list
: the entry will be used for all requests.
- Leading and trailing slashes in the starting portion are ignored, and / and \ are treated equivalently. Case is also ignored.
- For local and remote directory entries, trailing slashes are ignored.
- Remote virtual directories are not supported for: server side includes, image maps, file uploads, CGI-BIN scripts, and SRE-Filter add-ons. Basically, remote virtual directories are only used when resolving aselector specified in GET method requests.
The usernames and passwords file (USER_FILE)
Each entry in the the USER_FILE (with default name USERS.IN) contains an optional host nickname, a username, password, and privilege information. It is used in two cases:
- When logon is required of a client.
- When information on privileges is needed.
The structure of the USER_FILE is:
host_nickname NAME PASSWORD PRIV1 PRIV2 ...
Note that the PRIV1 is required, while other privileges (PRIV2, etc.) are optional. Also note that the host_nickname is optionalExamples
DANIEL A11S34WZ SUPERUSER MIKE IOWAMAN CONSULTANT JILL GOBOTS INHOUSE JOHN KIOWA INHOUSE VENUS FLOWERS// CHARLES BROWN PRICELIST ANONYMOUS * PUBLICIn the above example:
- DANIEL is given SUPERUSER privileges. This could be useful if DANIEL needs to reset the server from an out-of-town site ... but it could be dangerous (so use an obscure password)!
- JILL and JOHN are IN-HOUSE users. Thus, if CHECKLOG=INHOUSE, then DANIEL, JILL, and JOHN are allowed in, but MIKE is not allowed in (MIKE is allowed in, given correct entry of his username and password, if CHECKLOG does not equal INHOUSE).
- JOHN is given VENUS privileges (as well as INHOUSE privileges).
- CHARLES has logon rights on the host with host nickname of FLOWERS (but not on any other host).
- ANONYMOUS is given PUBLIC privileges, and can use any password.
Note that * means "anything matches", and can be used in either the password or username fields.
ALIASES file (ALIAS_FILE)
The list of ALIASES is contained in the ALIAS_FILE (with default name ALIASES.IN)The ALIAS_FILE contains lines having the structure:
host_nickname alias_target replace_stringwhere host_nickname andalias_target
should have no embedded spaces.
Note that the host_nickname is optional.To support several special features, several special character strings may be placed in either the
alias_target
orreplace_string
:
- The * character in the
alias_target
signals that thisalias_target
can be used in "wildcard matching"- If a * appears in both the
replace_string
and thealias_target
, then an "wildcard match with substitution" may be performed.- If the
replace_string
starts with!TRANSFER
, then the (rest of) thereplace_string
is assumed to be an fully qualified file name (that is "not necessarily in the data directory"). Note that this !TRANFER syntax provides a limited alternative to "local" virtual directories- If the
replace_string
starts withhttp://
, then the remainder of thereplace_string
is assumed to be the URL for a redirection response to the client. Similarly, ifreplace_string
starts with !TEMP, !NOTIFY, or !MOVED, a "temporary redirection", "notification of moved URL", or "permanent" redirection response will occur.Examples
; This first example replaces a "bad request selector " with ;a correct HTML file names HOME INDEX.HTM ; The next example would be used if CHECKCAT.HTM was a gateway to a ; "searchable index" (CAT.DAT being the file that is actually searched). CHECKCAT.HTM?* DOSEARCH?FILE=CAT.DAT&SEARCH=* ; This redirects all request selectors that begin with YAH to yahoo YAH* http://www.yahoo.com/ ; This converts request selectors only to a host with a host_nickname of ANIMALS, ANIMALS// CATS/* FELINES/DOMESTIC/* ; This sends some "not in data directory" documentation to the client. WHATGOT !TRANSFER=E:\PRODUCTS\DESCRIBE.DOC ; This performs a "wildcard match with substitution, in combination with ; a redirection. Typically, this would be called as WEBSEARCH?find_this WEBSEARCH?* http://www.webcrawler.com/cgi-bin/WebQuery/?*CAUTION
Unless you explicitly want to "alias" selectors that otherwise would map to actual files,we recommend that all "wildcard match aliases" have "targets" that do NOT map to pre-existing files.The point is that a "wildcard matching" alias match could convert a legitimate selector (that points to a real file in the data directory or a virtual directory) into something else -- rendering it impossible to obtain this "real file".
Summary
Known limitations of SRE-Filter
The currently known limitations of the SRE-Filter package include:In other words, SRE-Filter is designed as relatively full-featured package for non-cutting edge small-to-medium load sites being maintained by non-professionals
- SRE-Filter has not been tested with proxy servers
- "Byte range retrieval" is tested, but should be viewed as experimental (GoServe can be a bit flakey when processing multi-part sends).
- SRE-Filter does not provide a high level of security -- there is no provision (currently) for data encryption.
- GoServe's caching is supported for most, but not all, file transfers. Caching is disallowed whenever a byte-range is retrieved, and when there is a possibility that a server side include will occur. Caching is also disallowed whenever logon or access control are active, unless a CACHE permission is enabled for this request selector .
SRE-Filter does provide two useful caches that in certain cases can be used instead of the GoServe cache: the server side include cache and the directory listing cacheWhen caching is on, and a cached file is requested, GoServe may call the filter anyways (see the Options-General tab of GoServe). If this should occur, SRE-Filter will detect that the file has already been transfered, and SRE-Filter will appropriately record the request (if RECORD_OPTION, WRITE_LOGS, or POST_FILTER is enabled).
Note that the COUNTER_FILE file will not be augmented (COUNTER_FILE is augmented via a server side include, and files with server side includes are never cached).- SRE-Filter 1.2i does not currently run under Object Rexx.
Summary of SRE-Filter
The following is a brief summary of SRE-Filter's features:
- Multiple hosts (either unique IP addresses, or multiple aliases per IP address) are supported.
- Seperate data directories, aliases, etc. can be specified for each supported host
- Logon requirements may be required of no one, everyone, or the general public.
- A set of public resources can be specifed, which will be available to all clients.
- OWNERS (with SUPERUSER privileges) and IN-HOUSE users can be automatically detected (using their numeric IP address), with logon requirments waived.
- A set of "unallowed" numeric IP addresses can be specified.
- Wildcards can be used in the list of IN-HOUSE and "unallowed" numeric IP addresses
- Several tools for resolving and redirecting requests are provided
- A "directory specific" document can be sent when a request selector specifying a directory (with no file is name) is recieved
- Instead of a default document; a "directory listing" can be produced, with file descriptions generated for all plain, html, and .ZIP files. These directory listings can be cached, and used for subsequent requests.
- ALIASes can be used to redirect requests for moved documents, to resolve commonly occuring "misspelled requests", and to transfer specific files from anywhere on your server.
- A customized "not found" document (complete with suggested link) can be sent when the requested document is missing.
- The ~ shorthand for the "home directory" is supported. In addition, ~ can point to www-specific subdirectories of the "home directory"
- Transfer of files from outside the "default data directory" is easily supported through the use of "virtual directories".
- Transfer, with optional processing, of files from remote servers (without using redirection) is supported through the use of "remote" virtual directories.
- A variety of server-side includes are easily achieved
- A special caching algorithim selectively caches documents that contain server side includes
- Headers and Footers can be included in all documents
- Recursively processed keyphrases, that dictate server side includes, can be placed in your documents
- Dynamic and Static String REPLACEments
- Inclusion of files (from anywhere on your system)
- Inclusion of OPTIONs that appear in the request selector (after a ?)
- Execution of REXX-code blocks, and insertion of the results
- SELECTive excludes of HTML code blocks
- The NSCA HTTPD server side include syntax is fully implemented
- Standard server features include:
- NCSA and CERN style IMAGEMAPS are supported, and easily requested.
- searchable indices on text files are easy to setup
- a flexible mechanism for controlling access to selected SEL-selectors (and the files they correspond to); including assignation of realms on a SEL-selector-specific basis.
- support for the HTACCESS method of access control
- several mechanisms for keeping track of file requests
- several forms of redirection can be specified
- a simple form of load balancing is supported
- support for "common-log" format audit files (as well as browser and referer log files)
- Special SRE-Filter built-in features include:
- a file display and transfer facility with easy directory traversal
- a simple but powerful "text file" search facility
- dynamic assignation of "short-term" privileges to clients
- tracking of recent hits to prevent recording of repetitive requests
- optional limits on frequency of unsuccesful "logon" attempts, with options for custom response files in the event of "logon" or "access" denial.
- A simple message-box facility is built into SRE-Filter (it can be used to support much more complex messaging packages).
- implementation of "byte range retrieval" (which is used by Adobe Acrobat to access subsections of documents)
- reponse headers can be automatially generated, using LINK and META HTTP-EQUIV elements in the <HEAD> portion of the document.
- Several file upload methods are supported
- Client directed retrieval of files from other HTTP servers
- Browsers that support the
type=file
FORM attribute (such as NetScape 2.01), can upload files from their own machines.- The PUT verb, with byte ranges, can be permitted on a per selector (or per "set of selectors ) basis
- The DELETE method can be permitted on a per selector (or per "set of selectors") basis
- User written server side processing (gateway programs) options include:
- Support for SRE-Filter add-ons (REXX routines that return results directly to the client).
- CGI-BIN scripts (both REXX and binary executables, and PERL scripts if you have a PERL interpreter installed) are supported (using procedures adapted from the GoHTTP package).
- A "pre-filter" can be called prior to invoking SRE-Filter
- SRE-Filter comes with a special pre-filter that implements the the GoRemote package.
- A "post-filter" can be called after SRE-Filter has responded to the request.
- SRE-Filter comes with a special post-filter that uses socket calls, to an SMTP server, to e-mail "event specific" alerts.
Terminology
The following terms are used in this documentation. Please forgive any non-standard uses you may encounter (better yet, let me know).You might also want to review the basic terminology for extended descriptions of a few key concepts.
- The \ and / characters
- The / character is used in URLs to "indicate hierarchical structures, such as directories". It is equivalent to the \ OS/2 (and DOS) character. Thus, / that appear in a request selector are converted into \ when SRE-Filter needs to find a file.
- Action
- The action is the "location" portion of a request selector.
- If the GET method was used -- it's the part after the first / (that ends the domain name) and before the first ? (or the end of the request selector , if no ? is present).
Examples:
- In
http://rpbcam.econ.ag.gov/calc/calc.htm
,
the action woud becalc/calc.htm
.- In:
http://foo.bar.net/workdir/myfile.htm?not+readyl
,
the action would be isworkdir/myfile.htm
.- In:
http://young.kids.com/toys/demo?dogs+cats
,
the action would betoys/demo
- If the POST method was used, the request selector (not including a leading / character) is the action (there should never be a ? in a POST method request string). All other information is contained in the "body" of the request. SRE-Filter assumes that all POST method requests are for Server Side Processing.
- Alias -- converting selectors
- SRE-Filter uses "aliases" to perform textual substitution. Typically, an alias is used to replace a simple request selector with a more complicated one. In addition, aliases are used for several special purposes: implementing "searchable indices", document redirections, and specification of the location of CGI-BIN scripts.
- Alias -- virtual hosts
- A given IP address can be used by several hosts: including the host associated with the "canonical" name, and hosts associated with one of several aliases. Unfortunately, TCP/IP requests arrive with only the numeric IP address, making it difficult for the server to determine to which host the request is directed to. However, HTTP 1.1 compliant browsers (such as NetScape 2.01) will include a
HOST:
request header containing the host to whom this request is directed; and SRE-Filter can use this request header (if present) to determine the proper host.- Client
- The Client (sometimes referred to as the requester) is the individual requesting a URL -- where the URL may point to a file, or may invoke Server Side Processing. Typically, the request was generated by someone running a Web Browser, and clicking on a link or submitting a FORM.
- CGI-BIN
- CGI (Common Gateway Interfact) - BIN is a standard by which server side ("gateway") programs can be invoked. It uses environment variables to pass data, as well as standard input and output. SRE-Filter supports CGI-BIN, using procedures adapted from the GoHTTP package. One nice feature of SRE-Filter is an ability to call alternate (non-REXX) interpreters for specific scripts (such as PERL when a script has a .PL extension).
Although this support should permit use of many CGI-BIN scripts, the use of the "native" SRE-Filter interface (using REXX procesures that are passed information in the argument list) offers advantages to ambitous programmers- The Default Data Directory
- The Default Data Directory is the root directory of your web site -- it (and it's subdirectories) are the usual locations for the HTML documents, images. In simple cases, the GoServe data directory is used as the the Default Data Directory. On multiple-hosts server, each host can be granted it's own Default Data Directory.
A simple example might help:
then, SRE-Filter will look in E:\WWW\MYFILES\PROJECT1 for the BIGTASK.HTM file.
- a request selector of MYFILES/PROJECT1/BIGTASK.HTM is recieved
- your default data directory is E:\WWW
A Note On Data Security in SRE-Filter
In general, the only files available for clients are those in (or under) the Data Directory. Exceptions to this rule include:- Files explicitly transferred by user-supplied executable code (i.e; REXX procedures). Note that user-written Rexx code may be invoked by a CGI-BIN script or a SRE-Filter add-on, or by INTERPRET, #EXEC and SELECT keyphrases.
- A !TRANSFER "alias" can be used to access files from anywhere on the server.
- Files specified by literal PUBLIC_URLS.
- Virtual directories can point to anywhere.
- ELEMENT
- ELEMENT's are the constituent parts of an HTML document. They dictate the structure, layout, and display characteristics of an HTML document. ELEMENTS are identified using HTML tags, where an HTML tag consists of text within < and > characters. Often, a start and stop tag are used.
For example:
<H1> Welcome </H1>
: defines a "level 1 Header"<A href="/morejunk/stuff.htm"> More info? </A>
: defines a link to a local document.- File Names: Fully qualified and Relatve
- SRE-Filter can treat File Names as "fully qualified" names or as "relative" names. Fully qualified names should be fully pathed -- containing the complete path to the file (sometimes the drive letter will be missing, in which case the GoServe drive is usually assumed). Relative names are assumed to be in subdirectories of some assumed default. For example, when a client asks for a file name, it is assumed to be relative to the data directory (or to a matching virtual directory).
- FORMS
- The HTML FORM element is used to create a fill-in form. A FORM element typically contains an ACTION attribute, as well as a number of elements that provide editable fields (for the client to fill in or to otherwise select). Typically a form is designed in tandem with a server-side program.
Although not the only way to generate a Server Side Processing Request (one can use sets of detailed URLS), FORMS are probably the easiest for the average client to deal with.- Hosts
- It is possible for one server machine to handle requests to multiple IP addresses. This "multiple hosting" (also refered to as multi-homing) can occur either by having the server handle several numeric IP addresses, or by the existence of multiple aliases for a single numeric IP address.
- Host nickname
- SRE-Filter uses host nicknames when identifying which host a parameter applies to. Besides being a convenience, it allows several hosts to point to identical resources (given that each host has the same host nickname).
- HTTP : Hyper Text Transfer Protocol
- The rules by which web-aware clients and servers communicate.
- HTTP method (also known as the VERB)
- SRE-Filter recognizes five HTTP methods: GET, POST, HEAD, DELETE, and PUT.
- GET requests contain all information (action name and optional parameter list) in the request string (with the parameter list placed after a ? character)
- POST requests contain only the action name in the request string (the "body" of the request contains the parameter list). Currently, SRE-Filter assumes that all POST requests are requests for Server Side Processing.
- HEAD method requests are sent "meta information" on the requested file, the file itself is not sent.
- PUT method requests are used to upload information to the file pointed to by a request selector . SRE-Filter will only honor PUT method requests if specific permissions are granted for that selector (as set in the ACCESS_FILE).
- DELETE method requests are used to delete (the file pointed) a requested selector . SRE-Filter will only honor DELETE method requests if specific permissions are granted for that selector (as set in the ACCESS_FILE).
- HTML
- Hyper Text Markup Language (HTML) is the protocol used to create documents designed for display by WWW browsers (such as Netscape, Mosaic, MS Internet Explorer,and IBM Web Explorer). Basically, HTML consists of a set of formatting commands, which are included in a file containing the text to be displayed.
Note that to be recognized by SRE-Filter as an HTML file, a file must have an .HTML, .HTM, .SHT, or a .SHTML extension (though it's easy to add extensions to this list).Although there is a formal HTML standard (currently 2.0, with 3.2 under discussion), it is not rigidly adhered to. In particular, NetScape has it's own extensions; many of which are not recognized by other browsers. For example, IBM's Web Explorer ver 1.03 recognizes the CENTER element, and the SIZE attribute of the HR element, but not the FRAMES element.- Imagemap
- An image map (sometimes called a mappable image) is an inline image which, when clicked with the mouse, returns a pixel location to the server. Typically, the server then translates the pixel location into a URL, and redirects the client to that URL (which might be on an all together different server). SRE-Filter supports image maps using the MAPIMAGE/ "indicator string", along with NCSA and CERN style "MAP" files to use in translating pixel locations to URL's.
Note that some browsers (such as NETSCAPE 2.0) support "client side mappable images", which require no action by the server (but which require an HTML element not understood by many other browsers).- IN-HOUSE
- IN-HOUSE refers to clients that are "in-house"; specifically, they have had INHOUSE privileges assigned to them. The idea is that one often wants to differentiate members of your "organization" from the rest of the world, and place fewer limits on what they can obtain from the server.
- Macrospace
- Marospace is best thought of as a global procedure cache used by OS/2. When REXX programs are asked to load external procedures, they first look in macrospace. For efficiency reasons, SRE-Filter makes extensive use of macrospace.
- Privileges
- SRE-Filter uses "privileges" to control access to selected resources. There are two classes of privileges:
Resource privileges can be ONE_OF privileges ("or" privileges") or MUST_HAVE privileges ("and" privileges). Client privileges may be static (assigned using INITFILT.80 or the USER_FILE), or may be dynamic (set on the fly as a function of the client's earlier requests).
- "resource" privileges which are assigned to server resources (such as files, or directories), using entries in the ACCESS_FILE.
- client privileges which are granted to clients (based on their IP address, or their explicit username/password).
- Request selector
- When a client sends a request to a server, it contains three components: the http method, the selector, and the http protocol.
For example: GET /FOO1/BAR.HTM HTTP/1.0
In this documentation, the term request selector refers to this middle component, with leading / stripped, and character decoding performed.
In the above example, the request selector would be FOO1/BAR.HTM.
Note that if a GET HTTP method was used, the request selector may contain information following a ? (for example, SHOWINFO?database=Prices&item=apples).- Response Header
- The response header contains "meta-information" about the requested document. GoServe automatically sends a few response headers (such as content length and file creation date). In addition, using the
LINK
andHTTP-EQUIV
tags in the <HEAD> portion of your document, you can instruct SRE-Filter to automatically add other response headers.- Searchable Index
- A searchable index is a document that points to a file that is to be searched. One can think of a searchable index as:
.. the front end to a gateway program, and the file you search as the database represented by the document.Typically, a searchable index contains the <ISINDEX> element.- SEL-specific
- SEL-specific means "a piece of information specific to a request selector . Clarifying, this means that SRE-Filter will use the value of the request selector to find information (such as resource privilege information). Thus, SEL-specific resource privileges really means "privileges associated with a particular request selector (coming up with terminology isn't easy ....)
Note: the SRE-Filter documentation sometimes uses the term URL-specific to mean SEL-specific (it's a holdover from an earlier revision of the documentation).
- Server Side Includes
- Server side includes refer to on-the-fly modifications of requested HTML documents. Inclusions can be static, in which case the use of server side includes is mostly to benefit the creation and maintenance of these documents. Dynamic includes refer to information that changes, such as the current time and date -- obviously, these can not be specified in the document a priori, but must be determined at the time of request.
- Server Side Processing
- Often, the server provides computational services to clients. An example of this is a client specified search of a database, with the results displayed using an 'attractive" HTML document (generated just for this search). Or, the client might request a map of some portion of the U.S, displaying a choice of geographic features. Such actions require the server to execute some kind of program (perhaps a REXX program, or perhaps some other executable).
SRE-Filter add-ons, and CGI-BIN scripts, are typical means by which Server Side Processing is provided to clients.- Server Side Processing Request
- A Server Side Processing request is, generically speaking, "a set of parameters, sent by a client, and an action name, that dictate Server Side Processing". Often these are generated by a client responding to a FORM element in an HTML document (alternatively, they may be "hand coded" into a URL included in a HTML document). Server Side Processing requests are recognized by:
- If the GET method was used: the presence of a ?options string after a "non-HTML" action name; where options contains the "set of parameters".
Notes:
- SRE-Filter assumes that, regardless of the presence of a ? in the request selector , all action names that end with a .HTM, .HTML, .SHT, or .SHTML are requests for HTML documents (that is, are not requests for server side processing),
- Actions which require no parameters must have a ? trailing the action name.
- Server side programs are sometimes referred to as gateway programs.
- If the request selector starts with CGI-BIN/ , then it's assumed to be a request for a CGI-BIN script.
- If the POST method was used, SRE-Filter assumes a Server Side Processing Request, with the request selector treated as the action (which identifies the program to run), and the "set of parameters" contained in the "body" of the request.
- STEM variables.
- STEM variables are REXX's way of implementing arrays. A STEM variable consists of a NAME.INDEX1.INDEX2 syntax (that is, with a . used to indicate "array indices"). SRE-Filter uses STEM variables for several of it's configuration parameters.
- SUPERUSER status
- Clients with SUPERUSER status are granted full privileges. Currently, this includes:
- Never having to logon (SUPERUSERs are INHOUSE users)
- Automatically granted CONTROL privileges (can perform remote server resets)
- File uploads are never blocked.
- URL: Uniform Resource Locator.
- URL's are a scheme for specifying Internet resources using a single line of printable ASCII characters. A URL should contain a protocol, domain name, port number (optional), the location of the resource, and an (optional) option list (following a ?).
In this documentation, the URL term connotes "a link coded into an HTML document".
Upon selecting such a link, the client sends back a request string that contains the verb (the HTTP method), the request selector , and the http protocol (typically, HTTP/1.0)
Example:given a URL of http://www.econ.ag.gov/calc/calc.htm
the resulting request selector would becalc/calc.htm
- URL character encoding
- Since several characters have very specific meanings in the HTTP protocol (such as the & and the > characters), and since spaces are not allowed in request strings, a set of character encoding rules are needed to transmit such characters.
Caution: When handcoding URL's with an option list following a ? and seperated by & characters, you must encode the & character.For example:
- INCORRECT: <A HREF="/GETAFILE?dir=PUBLIC/&showdate=YES">
- CORRECT: <A HREF="/GETAFILE?dir=PUBLIC/&showdate=YES">
- Web Browser
- A Web Browser is a program that can communicate over the Internet, and that is capable of displaying HTML documents. NetScape, Mosaic. and IBM Web Explorer are examples of Web Browsers.
- Wildcard matching
- Wildcard matching use the * as a wildcard character in a natural fashion.
Examples:/JOE/* will match /JOE/FOO.HTM /JOAN/SRCH.HTM?* will match /JOAN/SRCH.HTM?search+me /JOAN/SRCH.HTM?* will NOT match /JOAN/SRCH.HTM (/JOAN/SRCH.HTM* will match BOTH these examples) /PETS/*INDEX.HTM will match /PETS/INDEX.HTM, /PETS/CAT/INDEX.HTM and /PETS/PUPPY/LAB/INDEX.HTM but will NOT match /PETS/CAT/PUREBRED.HTMSRE-Filter files that support wildcard matching are: ALIAS_FILE, RECORD_ALL_FILE, SENDFILE_FILE, ACCESS_FILE, and the PUBLIC_URL stem variables.
The COUNTER_FILE does not support wildcard matching.- Wildcard matching with substitution
- Wildcard matching with substitution is similar to wildcard matching. However, rather then being a "many to one" match, it is a "many to many" match.
The basic rule is that both a "target" and a "replacement" should contain * characters. When a wildcard match between the target and a candidate occurs, the "covered portion of the candidate" is inserted into the "replacement". In other words, the * character in the replacement is deleted, and the "covered portion" is inserted.Example:
- If the request selector is:
project/jillwork.htm
- and there is an alias of:
project/* /research/ongoing/*
- SRE-Filter will convert the request selector to
/research/ongoing/jillwork.htm
- Working Directory
- The Working Directory in the directory containing GOSERVE.EXE (typically, \GOSERV). You should copy the various SRE-Filter files to it (see the installation section for details).
Return to table of contents
For questions, contact Daniel Hellerstein.